Publications

• [ 2025 ]
• [ 2024 ]
• [ 2023 ]
• [ 2022 ]
• [ 2021 ]
• [ 2020 ]
• [ 2019 ]
• [ 2018 ]
• [ 2017 ]
• [ 2016 ]
• [ 2015 ]
• [ 2014 ]
• [ 2013 ]
• [ 2012 ]
• [ 2011 ]
• [ 2010 ]
• [ 2009 ]
• [ 2008 ]
• [ 2007 ]
• [ 2006 ]
• [ 2005 ]
• [ 2004 ]
• [ 2003 ]
• [ 2002 ]
• [ 2001 ]
• [ 1999 ]

• 
  Trust your local scaler: A continuous, decentralized approach to autoscaling. M. Straesser; S. Geissler; S. Lange; L. K. Schumann; T. Hossfeld; S. Kounev; in Performance Evaluation (2025). 167 102452.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.

  @article{STRAESSER2025102452, abstract = {Autoscaling is a critical component of modern cloud computing environments, improving flexibility, efficiency, and cost-effectiveness. Current approaches use centralized autoscalers that make decisions based on averaged monitoring data of managed service instances in fixed intervals. In this scheme, autoscalers are single points of failure, tightly coupled to monitoring systems, and limited in reaction times, making non-optimal scaling decisions costly. This paper presents an approach for continuous decentralized autoscaling, where decisions are made on a service instance level. By distributing scaling decisions of instances over time, autoscaling evolves into a quasi-continuous process, enabling great adaptability to different workload patterns. We analyze our approach on different abstraction levels, including a model-based, simulation-based, and real-world evaluation. Proof-of-concept experiments show that our approach is able to scale different applications deployed in containers and virtual machines in realistic environments, yielding better scaling performance compared to established baseline autoscalers, especially in scenarios with highly dynamic workloads.}, author = {Straesser, Martin and Geissler, Stefan and Lange, Stanislav and Schumann, Lukas Kilian and Hossfeld, Tobias and Kounev, Samuel}, journal = {Performance Evaluation}, keywords = {t_journalmagazine}, pages = 102452, title = {Trust your local scaler: A continuous, decentralized approach to autoscaling}, volume = 167, year = 2025 }

• 
  An Empirical Study on the Impact of Selected Host Configuration Parameters on Container Start Times. M. Straesser; N. Erhard; S. Kounev; (2024).

  In print.
  • [ BibTeX ]
  • [ Download ]
  @misc{StraesserSSP2024, author = {Straesser, Martin and Erhard, Nicholas and Kounev, Samuel}, booktitle = {Softwaretechnik-Trends Band 44, Heft 4}, keywords = {t_workshop}, note = {In print.}, publisher = {Gesellschaft für Informatik e.V.}, series = {Softwaretechnik-Trends}, title = {An Empirical Study on the Impact of Selected Host Configuration Parameters on Container Start Times}, year = 2024 }
• 
  Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis. M. Stenger; A. Bauer; T. Prantl; R. Leppich; N. Hudson; K. Chard; I. Foster; S. Kounev; in J. Data and Information Quality (2024). 16(2)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.

  @article{10.1145/3666006, abstract = {Time series data are widely used and provide a wealth of information for countless applications. However, some applications are faced with a limited amount of data, or the data cannot be used due to confidentiality concerns. To overcome these obstacles, time series can be generated synthetically. For example, electrocardiograms can be synthesized to make them available for building models to predict conditions such as cardiac arrhythmia without leaking patient information. Although many different approaches to time series synthesis have been proposed, evaluating the quality of synthetic time series data poses unique challenges and remains an open problem, as there is a lack of a clear definition of what constitutes a “good” synthesis. To this end, we present a comprehensive literature survey to identify different aspects of synthesis quality and their relationships. Based on this, we propose a definition of synthesis quality and a systematic evaluation procedure for assessing it. With this work, we aim to provide a common language and criteria for evaluating synthetic time series data. Our goal is to promote more rigorous and reproducible research in time series synthesis by enabling researchers and practitioners to generate high-quality synthetic time series data.}, address = {New York, NY, USA}, author = {Stenger, Michael and Bauer, Andr\'{e} and Prantl, Thomas and Leppich, Robert and Hudson, Nathaniel and Chard, Kyle and Foster, Ian and Kounev, Samuel}, journal = {J. Data and Information Quality}, keywords = {t_journalmagazine}, month = {06}, number = 2, publisher = {Association for Computing Machinery}, title = {Thinking in Categories: A Survey on Assessing the Quality for Time Series Synthesis}, volume = 16, year = 2024 }
• 
  Autoscaling Rethought: A Continuous, Decentralized Approach M. Straesser; (2024).
  • [ BibTeX ]
  • [ Download ]
  @presentation{straesser2024rethought, author = {Straesser, Martin}, keywords = {microservices}, title = {Autoscaling Rethought: A Continuous, Decentralized Approach}, year = 2024 }
• 
  Automatic Adversarial Adaption for Stealthy Poisoning Attacks in Federated Learning. T. Krauß; J. König; A. Dmitrienko; C. Kanzow; in To appear soon at the Network and Distributed System Security Symposium (NDSS) (2024).
  • [ BibTeX ]
  @article{krauss2024automatic, author = {Krauß, Torsten and König, Jan and Dmitrienko, Alexandra and Kanzow, Christian}, journal = {To appear soon at the Network and Distributed System Security Symposium (NDSS)}, keywords = {jankounig}, title = {Automatic Adversarial Adaption for Stealthy Poisoning Attacks in Federated Learning}, year = 2024 }
• 
  Beyond-local neural information processing in neuronal networks. J. Balkenhol; B. Händel; S. Biswas; J. Grohmann; J. v. Kistowski; J. Prada; C. A. Bosman; H. Ehrenreich; S. M. Wojcik; S. Kounev; R. Blum; T. Dandekar; in Computational and Structural Biotechnology Journal (2024).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  While there is much knowledge about local neuronal circuitry, considerably less is known about how neuronal input is integrated and combined across neuronal networks to encode higher order brain functions. One challenge lies in the large number of complex neural interactions. Neural networks use oscillating activity for information exchange between distributed nodes. To better understand building principles underlying the observation of synchronized oscillatory activity in a large-scale network, we developed a reductionistic neuronal network model. Fundamental building principles are laterally and temporally interconnected virtual nodes (microcircuits), wherein each node was modeled as a local oscillator. By this building principle, the neuronal network model can integrate information in time and space. The simulation gives rise to a wave interference pattern that spreads over all simulated columns in form of a travelling wave. The model design stabilizes states of efficient information processing across all participating neuronal equivalents. Model-specific oscillatory patterns, generated by complex input stimuli, were similar to electrophysiological high-frequency signals that we could confirm in the primate visual cortex during a visual perception task. Important oscillatory model pre-runners, limitations and strength of our reductionistic model are discussed. Our simple scalable model shows unique integration properties and successfully reproduces a variety of biological phenomena such as harmonics, coherence patterns, frequency-speed relationships, and oscillatory activities. We suggest that our scalable model simulates aspects of a basic building principle underlying oscillatory, large-scale integration of information in small and large brains.

  @article{BALKENHOL2024, abstract = {While there is much knowledge about local neuronal circuitry, considerably less is known about how neuronal input is integrated and combined across neuronal networks to encode higher order brain functions. One challenge lies in the large number of complex neural interactions. Neural networks use oscillating activity for information exchange between distributed nodes. To better understand building principles underlying the observation of synchronized oscillatory activity in a large-scale network, we developed a reductionistic neuronal network model. Fundamental building principles are laterally and temporally interconnected virtual nodes (microcircuits), wherein each node was modeled as a local oscillator. By this building principle, the neuronal network model can integrate information in time and space. The simulation gives rise to a wave interference pattern that spreads over all simulated columns in form of a travelling wave. The model design stabilizes states of efficient information processing across all participating neuronal equivalents. Model-specific oscillatory patterns, generated by complex input stimuli, were similar to electrophysiological high-frequency signals that we could confirm in the primate visual cortex during a visual perception task. Important oscillatory model pre-runners, limitations and strength of our reductionistic model are discussed. Our simple scalable model shows unique integration properties and successfully reproduces a variety of biological phenomena such as harmonics, coherence patterns, frequency-speed relationships, and oscillatory activities. We suggest that our scalable model simulates aspects of a basic building principle underlying oscillatory, large-scale integration of information in small and large brains.}, author = {Balkenhol, Johannes and Händel, Barbara and Biswas, Sounak and Grohmann, Johannes and v. Kistowski, Jóakim and Prada, Juan and Bosman, Conrado A. and Ehrenreich, Hannelore and Wojcik, Sonja M. and Kounev, Samuel and Blum, Robert and Dandekar, Thomas}, journal = {Computational and Structural Biotechnology Journal}, keywords = {t_journalmagazine}, title = {Beyond-local neural information processing in neuronal networks}, year = 2024 }
• 
  Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis. A. Bauer; T. Dittus; M. Straesser; A. Kamatar; M. Baughman; L. Beierlieb; M. Hadry; D. Grillmeyer; Y. Lubas; S. Kounev; I. Foster; K. Chard; in Companion of the 15th ACM/SPEC International Conference on Performance Engineering (2024). 72–76.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

  @inproceedings{10.1145/3629527.3651843, abstract = {As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.}, address = {New York, NY, USA}, author = {Bauer, André and Dittus, Timo and Straesser, Martin and Kamatar, Alok and Baughman, Matt and Beierlieb, Lukas and Hadry, Marius and Grillmeyer, Daniel and Lubas, Yannik and Kounev, Samuel and Foster, Ian and Chard, Kyle}, booktitle = {Companion of the 15th ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {72–76}, publisher = {Association for Computing Machinery}, series = {ICPE '24 Companion}, title = {Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis}, year = 2024 }
• 
  Triple-S: Security Scoring System for IoT Devices. M. Finke; A. Dmitrienko; in To be published at IEEE COINS (2024).
  • [ BibTeX ]
  @article{finke2024triples, author = {Finke, Moritz and Dmitrienko, Alexandra}, journal = {To be published at IEEE COINS}, keywords = {sssgroup}, title = {Triple-S: Security Scoring System for IoT Devices}, year = 2024 }
• 
  MirageFlow: A New Bandwidth Inflation Attack on Tor. C. Sendner; J. Stang; A. Dmitrienko; R. Wijewickrama; M. Jadliwala; in To appear soon at the Network and Distributed System Security Symposium (NDSS) (2024).
  • [ BibTeX ]
  @article{sendner2024mirageflow, author = {Sendner, Christoph and Stang, Jasper and Dmitrienko, Alexandra and Wijewickrama, Raveen and Jadliwala, Murtuza}, journal = {To appear soon at the Network and Distributed System Security Symposium (NDSS)}, keywords = {sss-group}, title = {MirageFlow: A New Bandwidth Inflation Attack on Tor}, year = 2024 }
• 
  MACRO: A Multi-Head Attentional Convolutional Recurrent Network for the Classification of Co-Occurring Diseases in 12-Lead ECGs. V. Borst; R. Leppich; S. Kounev; in 2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI) (2024).

  Acceptance Rate: 24.4%
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{borst2024MACRO, author = {Borst, Vanessa and Leppich, Robert and Kounev, Samuel}, booktitle = {2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI)}, keywords = {t_interdisciplinary}, note = {Acceptance Rate: 24.4%}, organization = {IEEE}, title = {{MACRO}: A Multi-Head Attentional Convolutional Recurrent Network for the Classification of Co-Occurring Diseases in 12-Lead {ECGs}}, year = 2024 }
• 
  Attack-Aware Security Function Chaining. L. Iffländer; L. Beierlieb; S. Kounev; in Electronics (2024). 13(17)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  Cyberattacks have become more frequent and more violent in recent years. To date, defensive infrastructure has been relatively static, and security functions are usually placed in a common order that does not depend on the current situation. We propose the concept of attack-aware Security Service Function Chain reordering. The idea is to change the order of security functions depending on the malicious traffic observed. We present the basic idea, evaluate the impact of the function chain order, and introduce a framework for function chain reordering. Our evaluation shows that the order often has a significant impact on the performance of the security function chain and that there is no single order that outperforms all other orders in every situation. The proposed proof-of-concept framework successfully validates the feasibility of attack-aware security function chain reordering, and we propose additional extensions to eliminate the remaining deficiencies.

  @article{electronics13173357, abstract = {Cyberattacks have become more frequent and more violent in recent years. To date, defensive infrastructure has been relatively static, and security functions are usually placed in a common order that does not depend on the current situation. We propose the concept of attack-aware Security Service Function Chain reordering. The idea is to change the order of security functions depending on the malicious traffic observed. We present the basic idea, evaluate the impact of the function chain order, and introduce a framework for function chain reordering. Our evaluation shows that the order often has a significant impact on the performance of the security function chain and that there is no single order that outperforms all other orders in every situation. The proposed proof-of-concept framework successfully validates the feasibility of attack-aware security function chain reordering, and we propose additional extensions to eliminate the remaining deficiencies.}, author = {Iffländer, Lukas and Beierlieb, Lukas and Kounev, Samuel}, journal = {Electronics}, keywords = {t_journalmagazine}, number = 17, title = {Attack-Aware Security Function Chaining}, volume = 13, year = 2024 }
• 
  ClearStamp: A Human-Visible and Robust Model-Ownership Proof based on Transposed Model Training. T. Krauß; J. Stang; A. Dmitrienko; in To Appear in the 33rd USENIX Security Symposium (USENIX Security 2024) (2024).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Due to costly efforts during data acquisition and model training, Deep Neural Networks (DNNs) belong to the intellectual property of the model creator. Hence, unauthorized use, theft, or modification may lead to legal repercussions. Existing DNN watermarking methods for ownership proof are often non-intuitive, embed human-invisible marks, require trust in algorithmic assessment that lacks human-understandable attributes, and rely on rigid thresholds, making it susceptible to failure in cases of partial watermark erasure. This paper introduces ClearStamp, the first DNN watermarking method designed for intuitive human assessment. ClearStamp embeds visible watermarks, enabling human decision-making without rigid value thresholds while allowing technology-assisted evaluations. ClearStamp defines a transposed model architecture allowing to use of the model in a backward fashion to interwove the watermark with the main task within all model parameters. Compared to existing watermarking methods, ClearStamp produces visual watermarks that are easy for humans to understand without requiring complex verification algorithms or strict thresholds. The watermark is embedded within all model parameters and entangled with the main task, exhibiting superior robustness. It shows an 8,544-bit watermark capacity comparable to the strongest existing work. Crucially, ClearStamp’s effectiveness is model and dataset-agnostic, and resilient against adversarial model manipulations, as demonstrated in a comprehensive study performed with four datasets and seven architectures.

  @article{torsten2024clearstamp, abstract = {Due to costly efforts during data acquisition and model training, Deep Neural Networks (DNNs) belong to the intellectual property of the model creator. Hence, unauthorized use, theft, or modification may lead to legal repercussions. Existing DNN watermarking methods for ownership proof are often non-intuitive, embed human-invisible marks, require trust in algorithmic assessment that lacks human-understandable attributes, and rely on rigid thresholds, making it susceptible to failure in cases of partial watermark erasure. This paper introduces ClearStamp, the first DNN watermarking method designed for intuitive human assessment. ClearStamp embeds visible watermarks, enabling human decision-making without rigid value thresholds while allowing technology-assisted evaluations. ClearStamp defines a transposed model architecture allowing to use of the model in a backward fashion to interwove the watermark with the main task within all model parameters. Compared to existing watermarking methods, ClearStamp produces visual watermarks that are easy for humans to understand without requiring complex verification algorithms or strict thresholds. The watermark is embedded within all model parameters and entangled with the main task, exhibiting superior robustness. It shows an 8,544-bit watermark capacity comparable to the strongest existing work. Crucially, ClearStamp’s effectiveness is model and dataset-agnostic, and resilient against adversarial model manipulations, as demonstrated in a comprehensive study performed with four datasets and seven architectures.}, author = {Krauß, Torsten and Stang, Jasper and Dmitrienko, Alexandra}, journal = {To Appear in the 33rd USENIX Security Symposium (USENIX Security 2024)}, keywords = {sssgroup}, title = {ClearStamp: A Human-Visible and Robust Model-Ownership Proof based on Transposed Model Training}, year = 2024 }
• 
  PUF-based Authentication in IoT against Strong Physical Adversary using Zero-Knowledge Proofs. L. Petzi; A. Dmitrienko; I. Visconti; in SafeThings (2024).
  • [ BibTeX ]
  @article{petzi2024pufbased, author = {Petzi, Lukas and Dmitrienko, Alexandra and Visconti, Ivan}, journal = {SafeThings}, keywords = {iot}, title = {PUF-based Authentication in IoT against Strong Physical Adversary using Zero-Knowledge Proofs}, year = 2024 }
• 
  Towards Explainable, Coordinated and Proactive Autoscaling for Microservices and Function Chains M. Straesser; (2024).
  • [ BibTeX ]
  • [ Download ]
  @presentation{straesser2024towards, author = {Straesser, Martin}, keywords = {microservices}, title = {Towards Explainable, Coordinated and Proactive Autoscaling for Microservices and Function Chains}, year = 2024 }
• 
  SPOQchain: Platform for Secure, Scalable, and Privacy-Preserving Supply Chain Tracing and Counterfeit Protection. M. Finke; A. Dmitrienko; J. Stang; in ArXiv | arXiv:2408.17049 (2024).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Product lifecycle tracing is increasingly in the focus of regulators and producers, as shown with the initiative of the Digital Product Pass. Likewise, new methods of counterfeit detection are developed that are, e.g., based on Physical Unclonable Functions (PUFs). In order to ensure trust and integrity of product lifecycle data, multiple existing supply chain tracing systems are built on blockchain technology. However, only few solutions employ secure identifiers such as PUFs. Furthermore, existing systems that publish the data of individual products, in part fully transparently, have a detrimental impact on scalability and the privacy of users. This work proposes SPOQchain, a novel blockchain-based platform that provides comprehensive lifecycle traceability and originality verification while ensuring high efficiency and user privacy. The improved efficiency is achieved by a sophisticated batching mechanism that removes lifecycle redundancies. In addition to the successful evaluation of SPOQchain's scalability, this work provides a comprehensive analysis of privacy and security aspects, demonstrating the need and qualification of SPOQchain for the future of supply chain tracing.

  @article{finke2024spoqchain, abstract = {Product lifecycle tracing is increasingly in the focus of regulators and producers, as shown with the initiative of the Digital Product Pass. Likewise, new methods of counterfeit detection are developed that are, e.g., based on Physical Unclonable Functions (PUFs). In order to ensure trust and integrity of product lifecycle data, multiple existing supply chain tracing systems are built on blockchain technology. However, only few solutions employ secure identifiers such as PUFs. Furthermore, existing systems that publish the data of individual products, in part fully transparently, have a detrimental impact on scalability and the privacy of users. This work proposes SPOQchain, a novel blockchain-based platform that provides comprehensive lifecycle traceability and originality verification while ensuring high efficiency and user privacy. The improved efficiency is achieved by a sophisticated batching mechanism that removes lifecycle redundancies. In addition to the successful evaluation of SPOQchain's scalability, this work provides a comprehensive analysis of privacy and security aspects, demonstrating the need and qualification of SPOQchain for the future of supply chain tracing.}, author = {Finke, Moritz and Dmitrienko, Alexandra and Stang, Jasper}, journal = {ArXiv | arXiv:2408.17049}, keywords = {sssgroup}, title = {SPOQchain: Platform for Secure, Scalable, and Privacy-Preserving Supply Chain Tracing and Counterfeit Protection}, year = 2024 }
• 
  Time-Aware Face Anti-Spoofing with Rotation Invariant Local Binary Patterns and Deep Learning. M. Finke; A. Dmitrienko; in arXiv:2408.14829 (2024).
  • [ BibTeX ]
  • [ URL ]
  @article{finke2024timeaware, author = {Finke, Moritz and Dmitrienko, Alexandra}, journal = {arXiv:2408.14829}, keywords = {sssgroup}, title = {Time-Aware Face Anti-Spoofing with Rotation Invariant Local Binary Patterns and Deep Learning}, year = 2024 }
• 
  Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices. V. Borst; T. Dittus; K. Müller; S. Kounev; in 47th German Conference on Artificial Intelligence - KI 2024 (2024).
  • [ BibTeX ]
  • [ DOI ]
  • [ Download ]
  @conference{borst2024early, author = {Borst, Vanessa and Dittus, Timo and Müller, Konstantin and Kounev, Samuel}, booktitle = {47th German Conference on Artificial Intelligence - KI 2024}, keywords = {t_interdisciplinary}, title = {Early Explorations of Lightweight Models for Wound Segmentation on Mobile Devices}, year = 2024 }
• 
  A Realistic Industrial Car Factory Digital Twin Simulation for Virtual Reality Applications and Robotic Training. F. Spiess; S. Roy; M. Hollmann; C. Cosway; S. Kalafatis; in 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE) (2024). 1419–1426.
  • [ BibTeX ]
  • [ DOI ]
  @inproceedings{spiess2024realistic, author = {Spiess, Florian and Roy, Swarnabha and Hollmann, Mattias and Cosway, Christopher and Kalafatis, Stavros}, booktitle = {2024 IEEE 20th International Conference on Automation Science and Engineering (CASE)}, keywords = {descartes}, pages = {1419-1426}, title = {A Realistic Industrial Car Factory Digital Twin Simulation for Virtual Reality Applications and Robotic Training}, year = 2024 }
• 
  Verify your Labels! Trustworthy Predictions and Datasets via Confidence Scores. T. Krauß; J. Stang; A. Dmitrienko; in To Appear in the 33rd USENIX Security Symposium (USENIX Security 2024) (2024).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Machine learning is a rapidly evolving technology with manifold benefits. At its core lies the mapping between samples and corresponding target labels (SL-Mappings). Such mappings can originate from labeled dataset samples or from prediction generated during model inference. The correctness of SL-Mappings is crucial, both during training and for model predictions, especially when considering poisoning attacks. Existing standalone works from the dataset cleaning and prediction confidence scoring domains lack a dual-use tool offering an SL-Mappings score, which is impractical. Moreover, these works have drawbacks, e.g., dependence on specific model architectures and reliance on large datasets, which may not be accessible, or lack a meaningful confidence score. In this paper, we introduce LabelTrust, a versatile tool designed to generate confidence scores for SL-Mappings. We propose pipelines facilitating dataset cleaning and confidence scoring, mitigating the limitations of existing standalone approaches from each domain. Thereby, LabelTrust leverages a Siamese network trained via few-shot learning, requiring minimal clean samples and is agnostic to datasets and model architectures. We demonstrate LabelTrust’s efficacy in detecting poisoning attacks within samples and predictions alike, with a modest one-time training overhead of 34.56 seconds and an evaluation time of less than 1 second per SL-Mapping.

  @article{krauss2024verify, abstract = {Machine learning is a rapidly evolving technology with manifold benefits. At its core lies the mapping between samples and corresponding target labels (SL-Mappings). Such mappings can originate from labeled dataset samples or from prediction generated during model inference. The correctness of SL-Mappings is crucial, both during training and for model predictions, especially when considering poisoning attacks. Existing standalone works from the dataset cleaning and prediction confidence scoring domains lack a dual-use tool offering an SL-Mappings score, which is impractical. Moreover, these works have drawbacks, e.g., dependence on specific model architectures and reliance on large datasets, which may not be accessible, or lack a meaningful confidence score. In this paper, we introduce LabelTrust, a versatile tool designed to generate confidence scores for SL-Mappings. We propose pipelines facilitating dataset cleaning and confidence scoring, mitigating the limitations of existing standalone approaches from each domain. Thereby, LabelTrust leverages a Siamese network trained via few-shot learning, requiring minimal clean samples and is agnostic to datasets and model architectures. We demonstrate LabelTrust’s efficacy in detecting poisoning attacks within samples and predictions alike, with a modest one-time training overhead of 34.56 seconds and an evaluation time of less than 1 second per SL-Mapping.}, author = {Krauß, Torsten and Stang, Jasper and Dmitrienko, Alexandra}, journal = {To Appear in the 33rd USENIX Security Symposium (USENIX Security 2024)}, keywords = {sssgroup}, title = {Verify your Labels! Trustworthy Predictions and Datasets via Confidence Scores}, year = 2024 }
• 
  An Empirical Investigation of Container Building Strategies and Warm Times to Reduce Cold Starts in Scientific Computing Serverless Functions. A. Bauer; M. Gonthier; R. Chard; H. Pan; D. Grzenda; M. Straesser; J. G. Pauloski; A. Kamatar; M. E. Baughman; N. Hudson; I. Foster; K. Chard; in 2024 IEEE 20th International Conference on e-Science (e-Science) (2024).
  • [ BibTeX ]
  @inproceedings{bauer2024coldstarts, author = {Bauer, André and Gonthier, Maxime and Chard, Ryan and Pan, Haochen and Grzenda, Daniel and Straesser, Martin and Pauloski, J. Gregory and Kamatar, Alok and Baughman, Matthew E. and Hudson, Nathaniel and Foster, Ian and Chard, Kyle}, booktitle = {2024 IEEE 20th International Conference on e-Science (e-Science)}, keywords = {Scientific_Computing}, title = {An Empirical Investigation of Container Building Strategies and Warm Times to Reduce Cold Starts in Scientific Computing Serverless Functions}, year = 2024 }
• 
  Benchmarking of Secure Group Communication schemes with Focus on IoT. T. Prantl; A. Bauer; S. Engel; L. Horn; C. Krupitzer; L. Iffl"ander; S. Kounev; in Discover Data (2024). 2(1)
  • [ BibTeX ]
  • [ URL ]
  @article{PrBaEnHoKrIfKo_Discover_Data_2024, author = {Prantl, Thomas and Bauer, André and Engel, Simon and Horn, Lukas and Krupitzer, Christian and Iffl"{a}nder, Lukas and Kounev, Samuel}, journal = {Discover Data}, keywords = {t_journalmagazine}, month = {05}, number = 1, publisher = {Springer Science and Business Media LLC}, title = {Benchmarking of Secure Group Communication schemes with Focus on IoT}, volume = 2, year = 2024 }
• 
  Evaluation is key: a survey on evaluation measures for synthetic time series. M. Stenger; R. Leppich; I. Foster; S. Kounev; A. Bauer; in Journal of Big Data (2024). 11 66.
  • [ BibTeX ]
  @article{Stenger2024, author = {Stenger, Michael and Leppich, Robert and Foster, Ian and Kounev, Samuel and Bauer, Andr{\'e}}, journal = {Journal of Big Data}, keywords = {t_journalmagazine}, month = {05}, pages = 66, title = {Evaluation is key: a survey on evaluation measures for synthetic time series}, volume = 11, year = 2024 }
• 
  DNNShield: Embedding Identifiers for Deep Neural Network Ownership Verification. J. Stang; T. Krauß; A. Dmitrienko; in arXiv:2403.06581 (2024).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The surge in popularity of machine learning (ML) has driven significant investments in training Deep Neural Networks (DNNs). However, these models that require resource-intensive training are vulnerable to theft and unauthorized use. This paper addresses this challenge by introducing DNNShield, a novel approach for DNN protection that integrates seamlessly before training. DNNShield embeds unique identifiers within the model architecture using specialized protection layers. These layers enable secure training and deployment while offering high resilience against various attacks, including fine-tuning, pruning, and adaptive adversarial attacks. Notably, our approach achieves this security with minimal performance and computational overhead (less than 5% runtime increase). We validate the effectiveness and efficiency of DNNShield through extensive evaluations across three datasets and four model architectures. This practical solution empowers developers to protect their DNNs and intellectual property rights.

  @article{jasper2024dnnshield, abstract = {The surge in popularity of machine learning (ML) has driven significant investments in training Deep Neural Networks (DNNs). However, these models that require resource-intensive training are vulnerable to theft and unauthorized use. This paper addresses this challenge by introducing DNNShield, a novel approach for DNN protection that integrates seamlessly before training. DNNShield embeds unique identifiers within the model architecture using specialized protection layers. These layers enable secure training and deployment while offering high resilience against various attacks, including fine-tuning, pruning, and adaptive adversarial attacks. Notably, our approach achieves this security with minimal performance and computational overhead (less than 5% runtime increase). We validate the effectiveness and efficiency of DNNShield through extensive evaluations across three datasets and four model architectures. This practical solution empowers developers to protect their DNNs and intellectual property rights.}, author = {Stang, Jasper and Krauß, Torsten and Dmitrienko, Alexandra}, journal = {arXiv:2403.06581}, keywords = {sssgroup}, month = {03}, title = {DNNShield: Embedding Identifiers for Deep Neural Network Ownership Verification}, year = 2024 }
• 
  A Data-Driven Model for Range Sensors. F. Spiess; N. Strobel; T. Kaupp; S. Kounev; in International Journal of Semantic Computing (2024). 1–18.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In this paper, an analysis of the precision of range sensors is presented. Light Detection and Ranging (LIDAR) data from three different sensors (HLS-LFCD-LDS, SICK TIM561, and Kinect V2), stereo data from the Realsense D435, and structured light data from both Kinect V1 for Xbox and the Xtion PRO Live were analyzed regarding the influence of range, incident angle to the surface, and surface material. In addition, a comparison with standard deviation models based on vendor-provided specifications was performed. We found that the vendor specifications are in general not specific enough to facilitate accurate sensor simulations. Therefore, we developed a data-driven model for range precision. Our model can be used to create realistic sensor simulations and to develop robot navigation algorithms weighing sensor range readings based on the precision.

  @article{doi:10.1142/S1793351X24300012, abstract = {In this paper, an analysis of the precision of range sensors is presented. Light Detection and Ranging (LIDAR) data from three different sensors (HLS-LFCD-LDS, SICK TIM561, and Kinect V2), stereo data from the Realsense D435, and structured light data from both Kinect V1 for Xbox and the Xtion PRO Live were analyzed regarding the influence of range, incident angle to the surface, and surface material. In addition, a comparison with standard deviation models based on vendor-provided specifications was performed. We found that the vendor specifications are in general not specific enough to facilitate accurate sensor simulations. Therefore, we developed a data-driven model for range precision. Our model can be used to create realistic sensor simulations and to develop robot navigation algorithms weighing sensor range readings based on the precision.}, author = {Spiess, Florian and Strobel, Norbert and Kaupp, Tobias and Kounev, Samuel}, journal = {International Journal of Semantic Computing}, keywords = {t_journalmagazine}, month = {02}, pages = {1-18}, title = {A Data-Driven Model for Range Sensors}, year = 2024 }
• 
  Kako: Mirai’s Vaccine Using Worm-Like Propagation Methods To Immunize Devices. F. Hack; Thesis; Bachelor Thesis. (2024, January).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In 2016, multiple large Distributed Denial of Service (DDoS) attacks were observed and attributed to a new botnet called Mirai [1]. Instead of personal computers and mobile devices, Mirai targets Internet of Things (IoT) devices like routers and surveillence cam- eras. As these devices are often equipped with insecure credentials by default [2], Mirai features a scanner component to identify devices and target them with a dictionary attack of common logins to gain access to those devices. This poses a threat, as many traditional methods of securing devices are not applicable to IoT, due to their limitations in com- puting power and memory [3]. White-hat approaches were proposed, based on the idea of infecting devices with a benign Mirai, which protects these devices against malicious variants [4]. However, there is a potential danger that users will not be able to access their devices after being protected with a white-hat. In this thesis, we present Kako, our proposed solution for a white-hat Mirai, which aims to address the issue of locking users out of their devices, while ensuring device security. Kako introduces a web-based dashboard that will list all vulnerable devices connected. When Kako is loaded onto a device with insecure credentials, they are automatically changed to secure it. The new login information can then be retrieved from the dashboard, among other data collected, such as the Medium Access Control (MAC) address and the Linux kernel version. The device manufacturer is automatically determined to make recognizing devices easier. Our approach was evaluated in a virtual environment and confirmed to be effective, as machines can no longer be infected with Mirai after being secured with Kako. It is also shown how Mirai’s scanning component can still be used with Kako to automatically secure vulnerable devices. We believe that Kako can be used and extended in further research, to also protect devices from newer incarnations of Mirai and similar botnets.

  @mastersthesis{hack2024mirais, abstract = {In 2016, multiple large Distributed Denial of Service (DDoS) attacks were observed and attributed to a new botnet called Mirai [1]. Instead of personal computers and mobile devices, Mirai targets Internet of Things (IoT) devices like routers and surveillence cam- eras. As these devices are often equipped with insecure credentials by default [2], Mirai features a scanner component to identify devices and target them with a dictionary attack of common logins to gain access to those devices. This poses a threat, as many traditional methods of securing devices are not applicable to IoT, due to their limitations in com- puting power and memory [3]. White-hat approaches were proposed, based on the idea of infecting devices with a benign Mirai, which protects these devices against malicious variants [4]. However, there is a potential danger that users will not be able to access their devices after being protected with a white-hat. In this thesis, we present Kako, our proposed solution for a white-hat Mirai, which aims to address the issue of locking users out of their devices, while ensuring device security. Kako introduces a web-based dashboard that will list all vulnerable devices connected. When Kako is loaded onto a device with insecure credentials, they are automatically changed to secure it. The new login information can then be retrieved from the dashboard, among other data collected, such as the Medium Access Control (MAC) address and the Linux kernel version. The device manufacturer is automatically determined to make recognizing devices easier. Our approach was evaluated in a virtual environment and confirmed to be effective, as machines can no longer be infected with Mirai after being secured with Kako. It is also shown how Mirai’s scanning component can still be used with Kako to automatically secure vulnerable devices. We believe that Kako can be used and extended in further research, to also protect devices from newer incarnations of Mirai and similar botnets.}, author = {Hack, Felix}, keywords = {sssgroup}, month = {01}, publisher = {Bachelor Thesis}, title = {Kako: Mirai’s Vaccine Using Worm-Like Propagation Methods To Immunize Devices}, type = {Bachelor Thesis}, year = 2024 }
• 
  Patient-Centered Chronic Wound Care Mobile Apps: Systematic Identification, Analysis, and Assessment. T. Dege; B. Glatzel; V. Borst; F. Grän; S. Goller; C. Glatzel; M. Goebeler; A. Schmieder; in JMIR mHealth and uHealth (2024). 12
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  Background: The prevalence of chronic wounds is predicted to increase within the aging populations in industrialized countries. Patients experience significant distress due to pain, wound secretions, and the resulting immobilization. As the number of wounds continues to rise, their adequate care becomes increasingly costly in terms of health care resources worldwide. eHealth support systems are being increasingly integrated into patient care. However, to date, no systematic analysis of such apps for chronic wounds has been published. Objective: The aims of this study were to systematically identify and subjectively assess publicly available German- or English-language mobile apps for patients with chronic wounds, with quality assessments performed by both patients and physicians. Methods: Two reviewers independently conducted a systematic search and assessment of German- or English-language mobile apps for patients with chronic wounds that were available in the Google Play Store and Apple App Store from April 2022 to May 2022. In total, 3 apps met the inclusion and exclusion criteria and were reviewed independently by 10 physicians using the German Mobile App Rating Scale (MARS) and the System Usability Scale (SUS). The app with the highest mean MARS score was subsequently reviewed by 11 patients with chronic wounds using the German user version of the MARS (uMARS) and the SUS. Additionally, Affinity for Technology Interaction (ATI) scale scores were collected from both patients and physicians. Results: This study assessed mobile apps for patients with chronic wounds that were selected from a pool of 118 identified apps. Of the 73 apps available in both app stores, 10 were patient oriented. After excluding apps with advertisements or costs, 3 apps were evaluated by 10 physicians. Mean MARS scores ranged from 2.64 (SD 0.65) to 3.88 (SD 0.65) out of 5, and mean SUS scores ranged from 50.75 (SD 27) to 80.5 (SD 17.7) out of 100. WUND APP received the highest mean MARS score (mean 3.88, SD 0.65 out of 5) among physicians. Hence, it was subsequently assessed by 11 patients and achieved a similar rating (uMARS score: mean 3.89, SD 0.4 out of 5). Technical affinity, as measured with the ATI scale, was slightly lower in patients (score: mean 3.62, SD 1.35 out of 6) compared to physicians (score: mean 3.88, SD 1.03 out 6). Conclusions: The quality ratings from physicians and patients were comparable and indicated mediocre app quality. Technical affinity, as assessed by using the ATI scale, was slightly lower for patients. Adequate apps for patients with chronic wounds remain limited, emphasizing the need for improved app development to meet patient needs. The ATI scale proved valuable for assessing technical affinity among different user groups.

  @article{dege2024patientcentered, abstract = {Background: The prevalence of chronic wounds is predicted to increase within the aging populations in industrialized countries. Patients experience significant distress due to pain, wound secretions, and the resulting immobilization. As the number of wounds continues to rise, their adequate care becomes increasingly costly in terms of health care resources worldwide. eHealth support systems are being increasingly integrated into patient care. However, to date, no systematic analysis of such apps for chronic wounds has been published. Objective: The aims of this study were to systematically identify and subjectively assess publicly available German- or English-language mobile apps for patients with chronic wounds, with quality assessments performed by both patients and physicians. Methods: Two reviewers independently conducted a systematic search and assessment of German- or English-language mobile apps for patients with chronic wounds that were available in the Google Play Store and Apple App Store from April 2022 to May 2022. In total, 3 apps met the inclusion and exclusion criteria and were reviewed independently by 10 physicians using the German Mobile App Rating Scale (MARS) and the System Usability Scale (SUS). The app with the highest mean MARS score was subsequently reviewed by 11 patients with chronic wounds using the German user version of the MARS (uMARS) and the SUS. Additionally, Affinity for Technology Interaction (ATI) scale scores were collected from both patients and physicians. Results: This study assessed mobile apps for patients with chronic wounds that were selected from a pool of 118 identified apps. Of the 73 apps available in both app stores, 10 were patient oriented. After excluding apps with advertisements or costs, 3 apps were evaluated by 10 physicians. Mean MARS scores ranged from 2.64 (SD 0.65) to 3.88 (SD 0.65) out of 5, and mean SUS scores ranged from 50.75 (SD 27) to 80.5 (SD 17.7) out of 100. WUND APP received the highest mean MARS score (mean 3.88, SD 0.65 out of 5) among physicians. Hence, it was subsequently assessed by 11 patients and achieved a similar rating (uMARS score: mean 3.89, SD 0.4 out of 5). Technical affinity, as measured with the ATI scale, was slightly lower in patients (score: mean 3.62, SD 1.35 out of 6) compared to physicians (score: mean 3.88, SD 1.03 out 6). Conclusions: The quality ratings from physicians and patients were comparable and indicated mediocre app quality. Technical affinity, as assessed by using the ATI scale, was slightly lower for patients. Adequate apps for patients with chronic wounds remain limited, emphasizing the need for improved app development to meet patient needs. The ATI scale proved valuable for assessing technical affinity among different user groups.}, author = {Dege, Tassilo and Glatzel, Bernadette and Borst, Vanessa and Grän, Franziska and Goller, Simon and Glatzel, Caroline and Goebeler, Matthias and Schmieder, Astrid}, journal = {JMIR mHealth and uHealth}, keywords = {t_journalmagazine}, month = {03}, title = {Patient-Centered Chronic Wound Care Mobile Apps: Systematic Identification, Analysis, and Assessment}, volume = 12, year = 2024 }
• 
  Simulating Microservice-based Architectures for Resilience Assessment Enriched by Authentic Container Orchestration. S. Frank; M. Straesser; L. Wagner; P. Haas; A. Hakamian; S. Kounev; A. Van Hoorn; in Software Engineering 2024 (2024).
  • [ BibTeX ]
  • [ Download ]
  @incollection{Frank2024Simulating, address = {Bonn}, author = {Frank, Sebastian and Straesser, Martin and Wagner, Lion and Haas, Patrick and Hakamian, Alireza and Kounev, Samuel and Van Hoorn, André}, booktitle = {Software Engineering 2024}, keywords = {container_orchestration}, publisher = {Gesellschaft für Informatik e.V.}, title = {Simulating Microservice-based Architectures for Resilience Assessment Enriched by Authentic Container Orchestration}, year = 2024 }
• 
  Comprehensive Exploration of Synthetic Data Generation: A Survey. A. Bauer; S. and Trapp; M. and Stenger; R. and Leppich; S. and Kounev; M. and Leznik; K. and Chard; I. and Foster; in arXiv preprint (2024). 103.

  https://doi.org/10.48550/arXiv.2401.02524
  • [ BibTeX ]
  @article{bauer2024comprehensive, author = {Bauer, André and and Trapp, Simon and and Stenger, Michael and and Leppich, Robert and and Kounev, Samuel and and Leznik, Mark and and Chard, Kyle and and Foster, Ian}, journal = {arXiv preprint}, keywords = {t_journalmagazine}, note = {https://doi.org/10.48550/arXiv.2401.02524}, pages = 103, title = {Comprehensive Exploration of Synthetic Data Generation: A Survey}, year = 2024 }
• 
  CrowdGuard: Federated Backdoor Detection in Federated Learning. T. Krauß; P. Rieger; M. Miettinen; A. Dmitrienko; A.-R. Sadeghi; in To appear soon at the Network and Distributed System Security Symposium (NDSS) (2024).
  • [ BibTeX ]
  @article{torsten2024crowdguard, author = {Krauß, Torsten and Rieger, Phillip and Miettinen, Markus and Dmitrienko, Alexandra and Sadeghi, Ahmad-Reza}, journal = {To appear soon at the Network and Distributed System Security Symposium (NDSS)}, keywords = {Private_AI}, title = {CrowdGuard: Federated Backdoor Detection in Federated Learning}, year = 2024 }
• 
  Time-Aware Face Anti-Spoofing with Rotation Invariant Local Binary Patterns and Deep Learning. M. Finke; A. Dmitrienko; in ArXiv | arXiv:2408.14829 (2024).
  • [ BibTeX ]
  • [ URL ]
  @article{noauthororeditor2024, author = {Finke, Moritz and Dmitrienko, Alexandra}, journal = {ArXiv | arXiv:2408.14829}, keywords = {facialauthentication}, title = {Time-Aware Face Anti-Spoofing with Rotation Invariant Local Binary Patterns and Deep Learning}, year = 2024 }
• 
  FreqFed: A Frequency Analysis-Based Approach for Mitigating Poisoning Attacks in Federated Learning. H. Fereidooni; A. Pegoraro; P. Rieger; A. Dmitrienko; A.-R. Sadeghi; in To appear soon at the Network and Distributed System Security Symposium (NDSS) (2024).
  • [ BibTeX ]
  @article{fereidooni2024freqfed, author = {Fereidooni, Hossein and Pegoraro, Alessandro and Rieger, Phillip and Dmitrienko, Alexandra and Sadeghi, Ahmad-Reza}, journal = {To appear soon at the Network and Distributed System Security Symposium (NDSS)}, keywords = {sss-group}, title = {FreqFed: A Frequency Analysis-Based Approach for Mitigating Poisoning Attacks in Federated Learning}, year = 2024 }
• 
  Cloud-Based Machine Learning Models as Covert Communication Channels. T. Krauß; J. Stang; A. Dmitrienko; in To appear in the 19th ACM ASIA Conference on Computer and Communications Security (ACM ASIACCS 2024) (2024).
  • [ BibTeX ]
  • [ Download ]
  @article{torsten2024cloudbased, author = {Krauß, Torsten and Stang, Jasper and Dmitrienko, Alexandra}, journal = {To appear in the 19th ACM ASIA Conference on Computer and Communications Security (ACM ASIACCS 2024)}, keywords = {sssgroup}, title = {Cloud-Based Machine Learning Models as Covert Communication Channels}, year = 2024 }
• 
  Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme. T. Prantl; M. Lauer; L. Horn; S. Engel; D. Dingel; S. Kounev; A. Bauer; C. Krupitzer; in Proceedings of the 19th International Conference on Availability, Reliability and Security (2024).
  • [ BibTeX ]
  @inproceedings{ARES_2024_ABE, author = {Prantl, Thomas and Lauer, Marco and Horn, Lukas and Engel, Simon and Dingel, David and Kounev, Samuel and Bauer, André and Krupitzer, Christian}, booktitle = {Proceedings of the 19th International Conference on Availability, Reliability and Security}, keywords = {descartes}, publisher = {ACM}, series = {ARES 2024}, title = {Security Analysis of a Decentralized, Revocable and Verifiable Attribute-Based Encryption Scheme}, year = 2024 }

• 
  G-Scan: Graph Neural Networks for Line-Level Vulnerability Identification in Smart Contracts. C. Sendner; R. Zhang; A. Hefter; A. Dmitrienko; F. Koushanfar; in arXiv:2307.08549 (2023).
  • [ BibTeX ]
  • [ URL ]
  @article{sendner2023gscan, author = {Sendner, Christoph and Zhang, Ruisi and Hefter, Alexander and Dmitrienko, Alexandra and Koushanfar, Farinaz}, journal = {arXiv:2307.08549}, keywords = {sss-group}, title = {G-Scan: Graph Neural Networks for Line-Level Vulnerability Identification in Smart Contracts}, year = 2023 }
• 
  Autoscaler Evaluation and Configuration: A Practitioner’s Guideline. M. Straesser; S. Eismann; J. von Kistowski; A. Bauer; S. Kounev; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 31–41.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.

  @inproceedings{straesser2023guideline, abstract = {Autoscalers are indispensable parts of modern cloud deployments and determine the service quality and cost of a cloud application in dynamic workloads. The configuration of an autoscaler strongly influences its performance and is also one of the biggest challenges and showstoppers for the practical applicability of many research autoscalers. Many proposed cloud experiment methodologies can only be partially applied in practice, and many autoscaling papers use custom evaluation methods and metrics. This paper presents a practical guideline for obtaining meaningful and interpretable results on autoscaler performance with reasonable overhead. We provide step-by-step instructions for defining realistic usage behaviors and traffic patterns. We divide the analysis of autoscaler performance into a qualitative antipattern-based analysis and a quantitative analysis. To demonstrate the applicability of our guideline, we conduct several experiments with a microservice of our industry partner in a realistic test environment.}, address = {New York, NY, USA}, author = {Straesser, Martin and Eismann, Simon and von Kistowski, J\'{o}akim and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {methodology}, pages = {31-41}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {Autoscaler Evaluation and Configuration: A Practitioner's Guideline}, year = 2023 }
• 
  Performance Engineering of Serverless Applications and Platforms. S. Eismann; Thesis; Universität Würzburg. (2023).
  • [ BibTeX ]
  • [ Download ]
  @phdthesis{Eismann2023, author = {Eismann, Simon}, keywords = {descartes}, school = {Universit{\"a}t W{\"u}rzburg}, title = {Performance Engineering of Serverless Applications and Platforms}, type = {doctoralthesis}, year = 2023 }
• 
  Towards Simulation-Driven Optimization of Container Orchestration Mechanisms T. Dittus; M. Straesser; S. Kounev; (2023).
  • [ BibTeX ]
  • [ Download ]
  @presentation{dittus2023simdrivenopt, author = {Dittus, Timo and Straesser, Martin and Kounev, Samuel}, keywords = {container_orchestration}, title = {Towards Simulation-Driven Optimization of Container Orchestration Mechanisms}, year = 2023 }
• 
  TorMult: Introducing a Novel Tor Bandwidth Inflation Attack. C. Sendner; J. Stang; A. Dmitrienko; R. Wijewickrama; M. Jadliwala; in ArXiv | arXiv.2307.08550 (2023).
  • [ BibTeX ]
  • [ URL ]
  @article{sendner2023tormult, author = {Sendner, Christoph and Stang, Jasper and Dmitrienko, Alexandra and Wijewickrama, Raveen and Jadliwala, Murtuza}, journal = {ArXiv | arXiv.2307.08550}, keywords = {sss-group}, title = {TorMult: Introducing a Novel Tor Bandwidth Inflation Attack}, year = 2023 }
• 
  Constrained Learning for Improved Attack Resilience in Federated Learning. T. Fella; (2023).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Federated Learning is an approach to machine learning where several participants train models locally, followed by aggregating these models into a combined model. This ap- proach has several advantages over classic, centralized machine learning, e.g., that the training data does not have to leave the client’s devices. Defenses against poisoning at- tacks on Federated Learning are an area of active research. Most existing approaches detect malicious models by focusing only on the adversarial side. This thesis goes a differ- ent way, by proposing changes to benign training on the client side, which lead to improved detection capabilities of existing defense mechanisms. We do this by constraining certain model characteristics, so-called metrics, in benign learning. This causes the benign models to be more similar to each other within a certain metric that is leveraged by the defense mechanism. The central parts of our research are an evaluation of how to best constrain malicious learning, evaluating, among others, techniques from Multi-Task Learning. We then evaluate the system’s capability of detecting malicious models and other effects it has on the behavior of the Federated Learning environment.

  @mastersthesis{fella2023constrained, abstract = {Federated Learning is an approach to machine learning where several participants train models locally, followed by aggregating these models into a combined model. This ap- proach has several advantages over classic, centralized machine learning, e.g., that the training data does not have to leave the client’s devices. Defenses against poisoning at- tacks on Federated Learning are an area of active research. Most existing approaches detect malicious models by focusing only on the adversarial side. This thesis goes a differ- ent way, by proposing changes to benign training on the client side, which lead to improved detection capabilities of existing defense mechanisms. We do this by constraining certain model characteristics, so-called metrics, in benign learning. This causes the benign models to be more similar to each other within a certain metric that is leveraged by the defense mechanism. The central parts of our research are an evaluation of how to best constrain malicious learning, evaluating, among others, techniques from Multi-Task Learning. We then evaluate the system’s capability of detecting malicious models and other effects it has on the behavior of the Federated Learning environment.}, author = {Fella, Tobias}, keywords = {sssgroup}, title = {Constrained Learning for Improved Attack Resilience in Federated Learning}, year = 2023 }
• 
  BLOCSIE - Benchmark for LOCalization in a Simulated Industrial Environment. F. Spiess; D. Bluemm; N. Strobel; T. Kaupp; S. Kounev; in 2023 21st International Conference on Advanced Robotics (ICAR) (2023). 577–584.
  • [ BibTeX ]
  @inproceedings{10406777, author = {Spiess, Florian and Bluemm, Daniel and Strobel, Norbert and Kaupp, Tobias and Kounev, Samuel}, booktitle = {2023 21st International Conference on Advanced Robotics (ICAR)}, keywords = {descartes}, pages = {577-584}, title = {BLOCSIE - Benchmark for LOCalization in a Simulated Industrial Environment}, year = 2023 }
• 
  An Empirical Study of Container Image Configurations and Their Impact on Start Times. M. Straesser; A. Bauer; R. Leppich; N. Herbst; K. Chard; I. Foster; S. Kounev; in 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid) (2023). 94–105.
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  @inproceedings{straesser2023starttime, author = {Straesser, Martin and Bauer, André and Leppich, Robert and Herbst, Nikolas and Chard, Kyle and Foster, Ian and Kounev, Samuel}, booktitle = {2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing (CCGrid)}, keywords = {empirical_study}, pages = {94-105}, publisher = {IEEE}, title = {An Empirical Study of Container Image Configurations and Their Impact on Start Times}, year = 2023 }
• 
  Penetration Testing of the eSano Platform. M. Schumacher; Thesis; Bachelor Thesis. (2023).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  We performed a penetration test on the eSano eHealth platform. The platform provides internet- and mobile-based interventions for mental health and chronic disease treatment. The primary objective of the pentesting was to enhance the security of the eSano platform, which in turn ensures the protection of sensitive patient data used and stored within it. The scope of the pentesting was one of three sub-platforms of eSano, as well as the API which connects all sub-platforms with the backend database. We employed a combination of OWASP resources and various tools to identify potential attack vectors, and subsequently prioritized them based on their impact, enabling us to concentrate our e↵orts on addressing the most critical vulnerabilities first. During the penetration testing of the eSano platform, we identified several vulnerabilities. These included multiple vulnerabilities with low and medium impact, such as unused endpoints, functionalities, and missing rate limits at certain points. In addition to these lower impact vulnerabilities, we also identified several critical issues. One critical issue involved a clickjacking attack that could trick users into performing unintended actions. We could also query whether an email address was associated with a platform user, which could lead to further attacks. A misconfiguration in the CORS protocol allowed third- party websites to make privileged requests about authenticated users. Finally, we were able to bypass the file extension restrictions and perform successful XSS, SSRF, and DoS attacks on the file upload functionality. This could potentially lead to compromised user data, unauthorized access to sensitive information, and potential denial of service. The findings of the pentesting have been shared with the developers of the eSano project, enabling them to address the identified vulnerabilities. This will make the platform more secure for data belonging to future users.

  @mastersthesis{schumacher2023penetration, abstract = {We performed a penetration test on the eSano eHealth platform. The platform provides internet- and mobile-based interventions for mental health and chronic disease treatment. The primary objective of the pentesting was to enhance the security of the eSano platform, which in turn ensures the protection of sensitive patient data used and stored within it. The scope of the pentesting was one of three sub-platforms of eSano, as well as the API which connects all sub-platforms with the backend database. We employed a combination of OWASP resources and various tools to identify potential attack vectors, and subsequently prioritized them based on their impact, enabling us to concentrate our e↵orts on addressing the most critical vulnerabilities first. During the penetration testing of the eSano platform, we identified several vulnerabilities. These included multiple vulnerabilities with low and medium impact, such as unused endpoints, functionalities, and missing rate limits at certain points. In addition to these lower impact vulnerabilities, we also identified several critical issues. One critical issue involved a clickjacking attack that could trick users into performing unintended actions. We could also query whether an email address was associated with a platform user, which could lead to further attacks. A misconfiguration in the CORS protocol allowed third- party websites to make privileged requests about authenticated users. Finally, we were able to bypass the file extension restrictions and perform successful XSS, SSRF, and DoS attacks on the file upload functionality. This could potentially lead to compromised user data, unauthorized access to sensitive information, and potential denial of service. The findings of the pentesting have been shared with the developers of the eSano project, enabling them to address the identified vulnerabilities. This will make the platform more secure for data belonging to future users.}, author = {Schumacher, Moritz}, keywords = {sssgroup}, publisher = {Bachelor Thesis}, title = {Penetration Testing of the eSano Platform}, type = {Bachelor Thesis}, year = 2023 }
• 
  Serverless Computing Revisited: Evolution, State-of-the-Art, and Performance Challenges S. Kounev; (2023).

  Joint Keynote at ICPE 2023 Workshops: LTB 2023 und FastContinuum 2023
  • [ BibTeX ]
  • [ URL ]
  @presentation{kounev2023serverless, author = {Kounev, Samuel}, keywords = {descartes}, note = {Joint Keynote at ICPE 2023 Workshops: LTB 2023 und FastContinuum 2023}, title = {Serverless Computing Revisited: Evolution, State-of-the-Art, and Performance Challenges}, year = 2023 }
• 
  Metadata-based Malware Detection on Android using Machine Learning. A. Hefter; C. Sendner; A. Dmitrienko; in ArXiv | arXiv.2307.08547 (2023).
  • [ BibTeX ]
  • [ URL ]
  @article{noauthororeditormetadatabased, author = {Hefter, Alexander and Sendner, Christoph and Dmitrienko, Alexandra}, journal = {ArXiv | arXiv.2307.08547}, keywords = {sss-group}, title = {Metadata-based Malware Detection on Android using Machine Learning}, year = 2023 }
• 
  Challenges and Future Directions in Efficiency Benchmarking (Vision Paper). M. Meissner; K.-D. Lange; S. Sharma; J. Arnold; A. Cragin; P. Galizia; M. Petrich; B. Zhang; S. Kounev; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 51–55.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  SPEC benchmarks are crucial contributors behind the improvement of server efficiency since 2007, given their role in making the power consumption and efficiency of servers transparent for government regulators, customers, and the manufacturers themselves.As the IT landscape experiences radical transformations, efficiency benchmarks need to be updated accordingly to generate results relevant to government regulators, manufacturers, and customers. In this paper, we outline current challenges efficiency benchmark developers are tackling and highlight recent technological developments the next generation of efficiency benchmarks should take into account.

  @inproceedings{10.1145/3578245.3585034, abstract = {SPEC benchmarks are crucial contributors behind the improvement of server efficiency since 2007, given their role in making the power consumption and efficiency of servers transparent for government regulators, customers, and the manufacturers themselves.As the IT landscape experiences radical transformations, efficiency benchmarks need to be updated accordingly to generate results relevant to government regulators, manufacturers, and customers. In this paper, we outline current challenges efficiency benchmark developers are tackling and highlight recent technological developments the next generation of efficiency benchmarks should take into account.}, address = {New York, NY, USA}, author = {Meissner, Maximilian and Lange, Klaus-Dieter and Sharma, Sanjay and Arnold, Jeremy and Cragin, Aaron and Galizia, Patrick and Petrich, Mike and Zhang, Bin and Kounev, Samuel}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {51–55}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Challenges and Future Directions in Efficiency Benchmarking (Vision Paper)}, year = 2023 }
• 
  Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs. A. Bauer; M. Straesser; M. Leznik; L. Beierlieb; M. Hadry; N. Hudson; K. Chard; S. Kounev; I. Foster; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 95–99.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.

  @inproceedings{10.1145/3578245.3584693, abstract = {Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.}, address = {New York, NY, USA}, author = {Bauer, Andr\'{e} and Straesser, Martin and Leznik, Mark and Beierlieb, Lukas and Hadry, Marius and Hudson, Nathaniel and Chard, Kyle and Kounev, Samuel and Foster, Ian}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {distance_measures}, pages = {95–99}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs}, year = 2023 }
• 
  ExDe: Design space exploration of scheduler architectures and mechanisms for serverless data-processing. S. Talluri; N. Herbst; C. Abad; T. De Matteis; A. Iosup; in Future Generation Computer Systems (2023).

  https://www.sciencedirect.com/science/article/pii/S0167739X23004211
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Serverless computing is increasingly used for data-processing applications in both science and business domains. At the core of serverless data-processing systems is the scheduler, which ensures dynamic decisions about task and data placement. Due to the variety of user, cluster, and workload properties, the design space for high-performance and cost-effective scheduling architectures and mechanisms is vast. The large design space is difficult to explore and characterize. To help the system designer disentangle this complexity, we present ExDe, a framework to systematically explore the design space of scheduling architectures and mechanisms. The framework includes a conceptual model and a simulator to assist in design space exploration. We use the framework, and real-world workloads, to characterize the performance of three scheduling architectures and two mechanisms. Our framework is open-source software available on Zenodo.

  @article{TALLURI2023, abstract = {Serverless computing is increasingly used for data-processing applications in both science and business domains. At the core of serverless data-processing systems is the scheduler, which ensures dynamic decisions about task and data placement. Due to the variety of user, cluster, and workload properties, the design space for high-performance and cost-effective scheduling architectures and mechanisms is vast. The large design space is difficult to explore and characterize. To help the system designer disentangle this complexity, we present ExDe, a framework to systematically explore the design space of scheduling architectures and mechanisms. The framework includes a conceptual model and a simulator to assist in design space exploration. We use the framework, and real-world workloads, to characterize the performance of three scheduling architectures and two mechanisms. Our framework is open-source software available on Zenodo.}, author = {Talluri, Sacheendra and Herbst, Nikolas and Abad, Cristina and {De Matteis}, Tiziano and Iosup, Alexandru}, journal = {Future Generation Computer Systems}, keywords = {t_journalmagazine}, note = {https://www.sciencedirect.com/science/article/pii/S0167739X23004211}, title = {ExDe: Design space exploration of scheduler architectures and mechanisms for serverless data-processing}, year = 2023 }
• 
  Recommendation of Secure Group Communication Schemes using Multi Objective Optimization. T. Prantl; A. Bauer; L. Iffländer; C. Krupitzer; S. Kounev; in International Journal of Information Security (2023).
  • [ BibTeX ]
  @article{PrBaIfKrKo_IJoIS_SGC_Pareto, author = {Prantl, Thomas and Bauer, André and Iffländer, Lukas and Krupitzer, Christian and Kounev, Samuel}, journal = {International Journal of Information Security}, keywords = {t_journalmagazine}, title = {Recommendation of Secure Group Communication Schemes using Multi Objective Optimization}, year = 2023 }
• 
  MESAS: Poisoning Defense for Federated Learning Resilient against Adaptive Attackers. K. Torsten; D. Alexandra; in To appear soon at the ACM Conference on Computer and Communications Security (CCS) (2023).
  • [ BibTeX ]
  @article{torsten2023mesas, author = {Torsten, Krauß and Alexandra, Dmitrienko}, journal = {To appear soon at the ACM Conference on Computer and Communications Security (CCS)}, keywords = {sss-group}, title = {MESAS: Poisoning Defense for Federated Learning Resilient against Adaptive Attackers}, year = 2023 }
• 
  Performance Impact Analysis of Homomorphic Encryption: A Case Study Using Linear Regression as an Example. T. Prantl; S. Engel; L. Horn; D. Kaiser; L. Iffländer; A. Bauer; C. Krupitzer; S. Kounev; in Information Security Practice and Experience, W. Meng, Z. Yan, V. Piuri (eds.) (2023). 284–298.
  • [ Abstract ]
  • [ BibTeX ]
  In recent years, the trend has increasingly been to store and process data in the cloud. However, this is based on the premise that cloud providers treat the data in a trustworthy manner. One way of using the data in the cloud without the provider having access to it is homomorphic encryption. However, since this encryption has only recently become practicable, analysis of its for practical applications is still in its infancy. Therefore, we investigate the performance of homomorphic encryption using a real-world application, namely linear regression. Our main finding is that although the homomorphic computation of linear regression is in the range of minutes and thus slower than in the non-homomorphic case, linear regression can be computed homomorphic and is therefore suitable for use cases where data security is the top priority.

  @inproceedings{PrEnHoDeIfBaKrKo_ISPEC_De_Bello, abstract = {In recent years, the trend has increasingly been to store and process data in the cloud. However, this is based on the premise that cloud providers treat the data in a trustworthy manner. One way of using the data in the cloud without the provider having access to it is homomorphic encryption. However, since this encryption has only recently become practicable, analysis of its for practical applications is still in its infancy. Therefore, we investigate the performance of homomorphic encryption using a real-world application, namely linear regression. Our main finding is that although the homomorphic computation of linear regression is in the range of minutes and thus slower than in the non-homomorphic case, linear regression can be computed homomorphic and is therefore suitable for use cases where data security is the top priority.}, address = {Singapore}, author = {Prantl, Thomas and Engel, Simon and Horn, Lukas and Kaiser, Dennis and Iffl\"ander, Lukas and Bauer, Andr\'e and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Information Security Practice and Experience}, editor = {Meng, Weizhi and Yan, Zheng and Piuri, Vincenzo}, keywords = {t_proceedings}, pages = {284–298}, publisher = {Springer Nature Singapore}, title = {Performance Impact Analysis of Homomorphic Encryption: A Case Study Using Linear Regression as an Example}, year = 2023 }
• 
  Ensuring Integrity of NVMe Offloaded Data in Large-Scale Machine Learning. R. Götz; (2023).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Machine learning is increasingly used in security-critical domains and therefore has become an attractive target for attackers. In a targeted poisoning attack, the machine learning model is trained to behave normally on benign input, however, when a certain trigger is present in the input, an attacker-chosen misbehavior is triggered. At the same time, neural networks are constantly increasing in size, especially in the natural language processing domain, because more parameters can achieve higher accuracy. In recent years, there have been impressive innovations that have led to networks such as the Megatron-Turing NLG 530B with 530 billion parameters. DeepSpeed is an open-source deep learning optimization library that enables the training of such large networks. One of many innovations it implements is a Non-Volatile Memory Express (NVMe) offload that increases memory efficiency by moving data from expensive GPU and CPU memory to cheap NVMe memory. However, this mechanism opens a potential attack surface that could be exploited to perform poisoning attacks. Therefore, this thesis investigates the security of the NVMe offload mechanism and improves it. To achieve this, first, untargeted poisoning attack scenarios are tested to show that the NVMe offload is actually vulnerable. Then a security extension, able to guarantee the integrity and freshness of the data offloaded to the NVMe is designed and extensively evaluated. During this process, various trade-offs between security and performance impact are carefully considered through the implementation and benchmarking of several different versions of the extension. Based on the experimental results, the security extension is then further improved. Furthermore, this thesis also investigates if multithreading and the use of previously generated hash tables can reduce the performance impact.

  @mastersthesis{gotz2023ensuring, abstract = {Machine learning is increasingly used in security-critical domains and therefore has become an attractive target for attackers. In a targeted poisoning attack, the machine learning model is trained to behave normally on benign input, however, when a certain trigger is present in the input, an attacker-chosen misbehavior is triggered. At the same time, neural networks are constantly increasing in size, especially in the natural language processing domain, because more parameters can achieve higher accuracy. In recent years, there have been impressive innovations that have led to networks such as the Megatron-Turing NLG 530B with 530 billion parameters. DeepSpeed is an open-source deep learning optimization library that enables the training of such large networks. One of many innovations it implements is a Non-Volatile Memory Express (NVMe) offload that increases memory efficiency by moving data from expensive GPU and CPU memory to cheap NVMe memory. However, this mechanism opens a potential attack surface that could be exploited to perform poisoning attacks. Therefore, this thesis investigates the security of the NVMe offload mechanism and improves it. To achieve this, first, untargeted poisoning attack scenarios are tested to show that the NVMe offload is actually vulnerable. Then a security extension, able to guarantee the integrity and freshness of the data offloaded to the NVMe is designed and extensively evaluated. During this process, various trade-offs between security and performance impact are carefully considered through the implementation and benchmarking of several different versions of the extension. Based on the experimental results, the security extension is then further improved. Furthermore, this thesis also investigates if multithreading and the use of previously generated hash tables can reduce the performance impact.}, author = {Götz, Raphael}, keywords = {sssgroup}, title = {Ensuring Integrity of NVMe Offloaded Data in Large-Scale Machine Learning}, year = 2023 }
• 
  Deep Neural Network Regression for Normalized Digital Surface Model Generation with Sentinel-2 Imagery. K. Müller; R. Leppich; C. Geiß; V. Borst; P. A. Pelizari; S. Kounev; in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing (2023). 1–14.
  • [ Abstract ]
  • [ BibTeX ]
  In recent history, normalized digital surface models (nDSMs) have been constantly gaining importance as a means to solve large-scale geographic problems. High-resolution surface models are precious, as they can provide detailed information for a specific area. However, measurements with a high resolution are time-consuming and costly. Only a few approaches exist to create high-resolution normalized digital surface models for extensive areas. This paper explores approaches to extract high-resolution nDSMs from low-resolution Sentinel-2 data, allowing us to derive large-scale models. We thereby utilize the advantages of Sentinel 2 being open access, having global coverage, and providing steady updates through a high repetition rate. Several deep-learning models are trained to overcome the gap in producing high-resolution surface maps from low-resolution input data. With U-Net as a base architecture, we extend the capabilities of our model by integrating tailored multiscale encoders with differently sized kernels in the convolution as well as conformed self-attention inside the skip connection gates. Using pixel-wise regression, our U-Net base models can achieve a mean height error of approximately two meters. Moreover, through our enhancements to the model architecture, we reduce the model error by more than seven percent.

  @article{10189905, abstract = {In recent history, normalized digital surface models (nDSMs) have been constantly gaining importance as a means to solve large-scale geographic problems. High-resolution surface models are precious, as they can provide detailed information for a specific area. However, measurements with a high resolution are time-consuming and costly. Only a few approaches exist to create high-resolution normalized digital surface models for extensive areas. This paper explores approaches to extract high-resolution nDSMs from low-resolution Sentinel-2 data, allowing us to derive large-scale models. We thereby utilize the advantages of Sentinel 2 being open access, having global coverage, and providing steady updates through a high repetition rate. Several deep-learning models are trained to overcome the gap in producing high-resolution surface maps from low-resolution input data. With U-Net as a base architecture, we extend the capabilities of our model by integrating tailored multiscale encoders with differently sized kernels in the convolution as well as conformed self-attention inside the skip connection gates. Using pixel-wise regression, our U-Net base models can achieve a mean height error of approximately two meters. Moreover, through our enhancements to the model architecture, we reduce the model error by more than seven percent.}, author = {Müller, Konstantin and Leppich, Robert and Geiß, Christian and Borst, Vanessa and Pelizari, Patrick Aravena and Kounev, Samuel}, journal = {IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing}, keywords = {t_journalmagazine}, pages = {1-14}, title = {Deep Neural Network Regression for Normalized Digital Surface Model Generation with Sentinel-2 Imagery}, year = 2023 }
• 
  Power to the Applications: The Vision of Continuous Decentralized Autoscaling. M. Straesser; S. Geißler; T. Hoßfeld; S. Kounev; in 2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW) (2023). 281–283.
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  @inproceedings{straesser2023decentral, author = {Straesser, Martin and Geißler, Stefan and Hoßfeld, Tobias and Kounev, Samuel}, booktitle = {2023 IEEE/ACM 23rd International Symposium on Cluster, Cloud and Internet Computing Workshops (CCGridW)}, keywords = {t_poster}, pages = {281-283}, publisher = {IEEE}, title = {Power to the Applications: The Vision of Continuous Decentralized Autoscaling}, year = 2023 }
• 
  Generating Microservice Applications for Performance Benchmarking Y. Lubas; M. Straesser; S. Kounev; (2023).
  • [ BibTeX ]
  • [ Download ]
  @presentation{lubas2023generating, author = {Lubas, Yannik and Straesser, Martin and Kounev, Samuel}, keywords = {microservices}, title = {Generating Microservice Applications for Performance Benchmarking}, year = 2023 }
• 
  Towards a Cryptography Encyclopedia: A Survey on Attribute-Based Encryption. T. Prantl; T. Zeck; L. Horn; L. Iffländer; A. Bauer; A. Dmitrienko; C. Krupitzer; S. Kounev; in Journal of Surveillance, Security and Safety (2023).
  • [ BibTeX ]
  • [ DOI ]
  @article{PrZeHoIfBaDmKrKo_ABESurvey, author = {Prantl, Thomas and Zeck, Timo and Horn, Lukas and Iffl\"ander, Lukas and Bauer, Andr\'e and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, journal = {Journal of Surveillance, Security and Safety}, keywords = {t_journalmagazine}, title = {Towards a Cryptography Encyclopedia: A Survey on Attribute-Based Encryption}, year = 2023 }
• 
  Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration. M. Straesser; P. Haas; S. Frank; A. Hakamian; A. Van Hoorn; S. Kounev; in Performance Evaluation Methodologies and Tools (2023).

  In print.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes' kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation's use cases and authenticity.

  @inproceedings{straesser2023kubernetesintheloop, abstract = {Microservices deployed and managed by container orchestration frameworks like Kubernetes are the bases of modern cloud applications. In microservice performance modeling and prediction, simulations provide a lightweight alternative to experimental analysis, which requires dedicated infrastructure and a laborious setup. However, existing simulators cannot run realistic scenarios, as performance-critical orchestration mechanisms (like scheduling or autoscaling) are manually modeled and can consequently not be represented in their full complexity and configuration space. This work combines a state-of-the-art simulation for microservice performance with Kubernetes container orchestration. Hereby, we include the original implementation of Kubernetes artifacts enabling realistic scenarios and testing of orchestration policies with low overhead. In two experiments with Kubernetes' kube-scheduler and cluster-autoscaler, we demonstrate that our framework can correctly handle different configurations of these orchestration mechanisms boosting both the simulation's use cases and authenticity.}, author = {Straesser, Martin and Haas, Patrick and Frank, Sebastian and Hakamian, Alireza and Van Hoorn, André and Kounev, Samuel}, booktitle = {Performance Evaluation Methodologies and Tools}, keywords = {discrete_event_simulation}, note = {In print.}, title = {Kubernetes-in-the-Loop: Enriching Microservice Simulation Through Authentic Container Orchestration}, year = 2023 }
• 
  A Systematic Approach for Benchmarking of Container Orchestration Frameworks. M. Straesser; J. Mathiasch; A. Bauer; S. Kounev; in Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 187–198.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.

  @inproceedings{straesser2023orchestration, abstract = {Container orchestration frameworks play a critical role in modern cloud computing paradigms such as cloud-native or serverless computing. They significantly impact the quality and cost of service deployment as they manage many performance-critical tasks such as container provisioning, scheduling, scaling, and networking. Consequently, a comprehensive performance assessment of container orchestration frameworks is essential. However, until now, there is no benchmarking approach that covers the many different tasks implemented in such platforms and supports evaluating different technology stacks. In this paper, we present a systematic approach that enables benchmarking of container orchestrators. Based on a definition of container orchestration, we define the core requirements and benchmarking scope for such platforms. Each requirement is then linked to metrics and measurement methods, and a benchmark architecture is proposed. With COFFEE, we introduce a benchmarking tool supporting the definition of complex test campaigns for container orchestration frameworks. We demonstrate the potential of our approach with case studies of the frameworks Kubernetes and Nomad in a self-hosted environment and on the Google Cloud Platform. The presented case studies focus on container startup times, crash recovery, rolling updates, and more.}, address = {New York, NY, USA}, author = {Straesser, Martin and Mathiasch, Jonas and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {container_orchestration}, pages = {187-198}, publisher = {Association for Computing Machinery}, series = {ICPE '23}, title = {A Systematic Approach for Benchmarking of Container Orchestration Frameworks}, year = 2023 }
• 
  Segmentierung und Verlaufskontrolle chronischer Wunden durch künstliche Intelligenz mithilfe einer mobilen App (WUNDERKINT Studie). V. Borst; T. Dege; F. Grän; S. Hamrouni; D. Michel; V. Hager; M. Goebeler; R. Leppich; S. Kounev; A. Schmieder; in Allergologie. JDDG: Journal der Deutschen Dermatologischen Gesellschaft (2023). (Vol. 21) 28–177.

  Poster Abstract (No. P322, pp. 175-176)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @incollection{borst_ddg23_v2, author = {Borst, Vanessa and Dege, Tassilo and Grän, Franziska and Hamrouni, Sandra and Michel, Daniel and Hager, Valerie and Goebeler, Matthias and Leppich, Robert and Kounev, Samuel and Schmieder, Astrid}, booktitle = {Allergologie. JDDG: Journal der Deutschen Dermatologischen Gesellschaft}, keywords = {t_poster}, month = {04}, note = {Poster Abstract (No. P322, pp. 175-176)}, number = {S1}, pages = {28-177}, publisher = {John Wiley & Sons Ltd.}, title = {Segmentierung und Verlaufskontrolle chronischer Wunden durch künstliche Intelligenz mithilfe einer mobilen App (WUNDERKINT Studie)}, volume = 21, year = 2023 }
• 
  Challenges in the development of the EO Exploitation Platform terrabyte. J. Eberle; M. Schwinger; J. Zeidler; in Big Data from Space, P. Soille, S. Lumnitz, S. Albani (eds.) (2023). 97–100.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The development of platforms for Earth Observation (EO) data analyses is increasing as the volume of EO data continues to grow. National and international organizations need to provide their users with a platform with data access to multi-source EO data as well as computational resources. In addition to storage and computational resources, user-friendly tools and web services must be provided to allow users to easily explore the data available on the platform and perform data processing and analysis at scale. This paper describes the challenges and architectural choices of the German Aerospace Center's platform `terrabyte` and provides recommendations for data and platform providers.

  @inproceedings{dlr199763, abstract = {The development of platforms for Earth Observation (EO) data analyses is increasing as the volume of EO data continues to grow. National and international organizations need to provide their users with a platform with data access to multi-source EO data as well as computational resources. In addition to storage and computational resources, user-friendly tools and web services must be provided to allow users to easily explore the data available on the platform and perform data processing and analysis at scale. This paper describes the challenges and architectural choices of the German Aerospace Center's platform `terrabyte` and provides recommendations for data and platform providers.}, author = {Eberle, Jonas and Schwinger, Maximilian and Zeidler, Julian}, booktitle = {Big Data from Space}, editor = {Soille, Pierre and Lumnitz, Stefanie and Albani, Sergio}, keywords = {t_interdisciplinary}, month = 11, pages = {97–100}, publisher = {Publications Office of the European Union}, title = {Challenges in the development of the EO Exploitation Platform terrabyte}, year = 2023 }
• 
  ClearMark: Intuitive and Robust Model Watermarking via Transposed Model Training. T. Krauß; J. Stang; D. Alexandra; in ArXiv | arXiv:2310.16453v1 (2023).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Due to costly efforts during data acquisition and model training, Deep Neural Networks (DNNs) belong to the intellectual property of the model creator. Hence, unauthorized use, theft, or modification may lead to legal repercussions. Existing DNN watermarking methods for ownership proof are often non-intuitive, embed human-invisible marks, require trust in algorithmic assessment that lacks human-understandable attributes, and rely on rigid thresholds, making it susceptible to failure in cases of partial watermark erasure. This paper introduces ClearMark, the first DNN watermarking method designed for intuitive human assessment. ClearMark embeds visible watermarks, enabling human decision-making without rigid value thresholds while allowing technology-assisted evaluations. ClearMark defines a transposed model architecture allowing to use of the model in a backward fashion to interwove the watermark with the main task within all model parameters. Compared to existing watermarking methods, ClearMark produces visual watermarks that are easy for humans to understand without requiring complex verification algorithms or strict thresholds. The watermark is embedded within all model parameters and entangled with the main task, exhibiting superior robustness. It shows an 8,544-bit watermark capacity comparable to the strongest existing work. Crucially, ClearMark's effectiveness is model and dataset-agnostic, and resilient against adversarial model manipulations, as demonstrated in a comprehensive study performed with four datasets and seven architectures.

  @article{krauss2023clearmark, abstract = {Due to costly efforts during data acquisition and model training, Deep Neural Networks (DNNs) belong to the intellectual property of the model creator. Hence, unauthorized use, theft, or modification may lead to legal repercussions. Existing DNN watermarking methods for ownership proof are often non-intuitive, embed human-invisible marks, require trust in algorithmic assessment that lacks human-understandable attributes, and rely on rigid thresholds, making it susceptible to failure in cases of partial watermark erasure. This paper introduces ClearMark, the first DNN watermarking method designed for intuitive human assessment. ClearMark embeds visible watermarks, enabling human decision-making without rigid value thresholds while allowing technology-assisted evaluations. ClearMark defines a transposed model architecture allowing to use of the model in a backward fashion to interwove the watermark with the main task within all model parameters. Compared to existing watermarking methods, ClearMark produces visual watermarks that are easy for humans to understand without requiring complex verification algorithms or strict thresholds. The watermark is embedded within all model parameters and entangled with the main task, exhibiting superior robustness. It shows an 8,544-bit watermark capacity comparable to the strongest existing work. Crucially, ClearMark's effectiveness is model and dataset-agnostic, and resilient against adversarial model manipulations, as demonstrated in a comprehensive study performed with four datasets and seven architectures.}, author = {Krauß, Torsten and Stang, Jasper and Alexandra, Dmitrienko}, journal = {ArXiv | arXiv:2310.16453v1}, keywords = {sss-group}, month = 10, title = {ClearMark: Intuitive and Robust Model Watermarking via Transposed Model Training}, year = 2023 }
• 
  CROSSCON: Interoperable IoT Security Stack - The RISC-V Opportunity. S. Pinto; M. Breskvar; T. Gomes; H. Koshutanski; A. Pasic; P. Krol; E. Amri; D. Puron; Z. Hornak; M. Rovieri; A. Dmitrienko; A.-R. Sadeghi; B. Crispo; (2023).
  • [ BibTeX ]
  @conference{sandro2023crosscon, author = {Pinto, Sandro and Breskvar, Matjaz and Gomes, Tiago and Koshutanski, Hristo and Pasic, Aljosa and Krol, Piotr and Amri, Emna and Puron, David and Hornak, Zoltan and Rovieri, Marco and Dmitrienko, Alexandra and Sadeghi, Ahmad-Reza and Crispo, Bruno}, keywords = {sss-group}, title = {CROSSCON: Interoperable IoT Security Stack - The RISC-V Opportunity}, year = 2023 }
• 
  Sybil Attack in Tor - Shedding Light to the Diversity Problem in Tor. D. Schreider; Thesis; Bachelor Thesis. (2023, August).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Tor is the most widely used anonymity communication network, which relies on voluntarily operated servers called relays to direct user traffic through its network. With an estimated daily user base of around 2 million, Tor stands as a widely adopted platform. However, the Tor network faces a significant challenge known as the ”Diversity Problem”, arising from a concentration of power, manifested through the trust and dependence placed upon a few dominant operators controlling a substantial portion of the network. For the analysis of this matter, we developed an Analysis Tool to aggregate and analyze Tor data. The Sybil attack within the Tor network includes two scenarios. (I) The attacker in- troduces additional relays into the network to increase their control and influence. The adversary’s objective in this scenario is to monitor a significant portion of the network traffic within Tor. (II) The attacker intentionally avoids associating the relays they man- age under a single entity, which is the typical practice expected from relay operators. As a result of this, they could potentially acquire advantageous visibility to observe connec- tions established via Tor by strategically placing their relays in critical positions of the circuit. Particularly concerning is the attacker’s capability to easily add relays to the Tor network and refrain from declaring relay ownership. The potential privacy risks for users emerge from both attack scenarios, with the combined effect being particularly concern- ing. Within this thesis, we extensively explore the Sybil attack to gain a comprehensive understanding of the factors that impact its effectiveness and how this attack affects the ”Diversity Problem”. By conducting a thorough analysis of the Sybil attack, we pinpoint the primary factors that shape its effectiveness. Moreover, we carry out simulations of the Sybil attack. An attacker who controls 100 Gbit/s of bandwidth could achieve up to a 40% likelihood of monitoring the exiting traffic or a 20% likelihood of observing incoming traffic within the Tor network. This capacity enables the attacker to potentially de-anonymize user connec- tions. We provide a proof of concept for the Sybil attack, critically analyzing the impact the attack can have on the Tor network. The rationale behind emphasizing this analysis is rooted in the absence of prior investigations into the Sybil attack’s effectiveness within the Tor ecosystem. Understanding the intricacies of the attack is essential to fortify Tor’s se- curity and preserve its mission of providing privacy and anonymity to its diverse user base. Furthermore, our aim extends to cultivating awareness among researchers and Tor users regarding the far-reaching implications of the ”Diversity Problem”, which subsequently affects the reliance and trust on a limited number of dominant entities.

  @mastersthesis{schreider2023sybil, abstract = {Tor is the most widely used anonymity communication network, which relies on voluntarily operated servers called relays to direct user traffic through its network. With an estimated daily user base of around 2 million, Tor stands as a widely adopted platform. However, the Tor network faces a significant challenge known as the ”Diversity Problem”, arising from a concentration of power, manifested through the trust and dependence placed upon a few dominant operators controlling a substantial portion of the network. For the analysis of this matter, we developed an Analysis Tool to aggregate and analyze Tor data. The Sybil attack within the Tor network includes two scenarios. (I) The attacker in- troduces additional relays into the network to increase their control and influence. The adversary’s objective in this scenario is to monitor a significant portion of the network traffic within Tor. (II) The attacker intentionally avoids associating the relays they man- age under a single entity, which is the typical practice expected from relay operators. As a result of this, they could potentially acquire advantageous visibility to observe connec- tions established via Tor by strategically placing their relays in critical positions of the circuit. Particularly concerning is the attacker’s capability to easily add relays to the Tor network and refrain from declaring relay ownership. The potential privacy risks for users emerge from both attack scenarios, with the combined effect being particularly concern- ing. Within this thesis, we extensively explore the Sybil attack to gain a comprehensive understanding of the factors that impact its effectiveness and how this attack affects the ”Diversity Problem”. By conducting a thorough analysis of the Sybil attack, we pinpoint the primary factors that shape its effectiveness. Moreover, we carry out simulations of the Sybil attack. An attacker who controls 100 Gbit/s of bandwidth could achieve up to a 40% likelihood of monitoring the exiting traffic or a 20% likelihood of observing incoming traffic within the Tor network. This capacity enables the attacker to potentially de-anonymize user connec- tions. We provide a proof of concept for the Sybil attack, critically analyzing the impact the attack can have on the Tor network. The rationale behind emphasizing this analysis is rooted in the absence of prior investigations into the Sybil attack’s effectiveness within the Tor ecosystem. Understanding the intricacies of the attack is essential to fortify Tor’s se- curity and preserve its mission of providing privacy and anonymity to its diverse user base. Furthermore, our aim extends to cultivating awareness among researchers and Tor users regarding the far-reaching implications of the ”Diversity Problem”, which subsequently affects the reliance and trust on a limited number of dominant entities.}, author = {Schreider, Dominik}, keywords = {sssgroup}, month = {08}, publisher = {Bachelor Thesis}, school = {University of Würzburg}, title = {Sybil Attack in Tor - Shedding Light to the Diversity Problem in Tor}, type = {Bachelor Thesis}, year = 2023 }
• 
  MVNOCoreSim: A Digital Twin for Virtualized IoT-centric Mobile Core Networks. S. Geissler; F. Wamser; W. Bauer; S. Gebert; S. Kounev; T. Hossfeld; in IEEE Internet of Things (2023). 10(15) 13974–13987.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  The rapid growth of connected devices has led to the implementation of IoT-centric platforms designed to provide connectivity for machine-to-machine type communication. Specifically, mobile virtual network operators (MVNOs) provide global service for IoT use cases by leveraging roaming in readily deployed physical networks. This global deployment of connected devices poses a significant challenge regarding the operation of centralized core networks with respect to dimensioning, scaling as well as system survivability in case of signaling incidents. Overload control mechanisms for IoT mobile networks offer a proactive solution for mitigating excessive signaling traffic from IoT devices in mobile core networks. However, global scaling and a heterogeneous composition of devices present network operators with additional challenges. To address these challenges, this work presents a detailed, protocol level simulation framework of a real-world IoT MVNO core network. We present both models for the expected IoT signaling load and the processing of signaling messages based on measurements in a live, production environment as well as a dedicated testbed. Finally, we present a case study on various overload mechanisms and identify critical performance characteristics to compare their performance. The results of this study categorize overload control mechanisms and shed light on the necessity for MVNOs to deal with the upcoming IoT traffic by defining appropriate overload control mechanisms in mobile core networks to ensure network survivability under unforeseen conditions.

  @article{geissler2023mvnocoresim, abstract = {The rapid growth of connected devices has led to the implementation of IoT-centric platforms designed to provide connectivity for machine-to-machine type communication. Specifically, mobile virtual network operators (MVNOs) provide global service for IoT use cases by leveraging roaming in readily deployed physical networks. This global deployment of connected devices poses a significant challenge regarding the operation of centralized core networks with respect to dimensioning, scaling as well as system survivability in case of signaling incidents. Overload control mechanisms for IoT mobile networks offer a proactive solution for mitigating excessive signaling traffic from IoT devices in mobile core networks. However, global scaling and a heterogeneous composition of devices present network operators with additional challenges. To address these challenges, this work presents a detailed, protocol level simulation framework of a real-world IoT MVNO core network. We present both models for the expected IoT signaling load and the processing of signaling messages based on measurements in a live, production environment as well as a dedicated testbed. Finally, we present a case study on various overload mechanisms and identify critical performance characteristics to compare their performance. The results of this study categorize overload control mechanisms and shed light on the necessity for MVNOs to deal with the upcoming IoT traffic by defining appropriate overload control mechanisms in mobile core networks to ensure network survivability under unforeseen conditions.}, author = {Geissler, Stefan and Wamser, Florian and Bauer, Wolfgang and Gebert, Steffen and Kounev, Samuel and Hossfeld, Tobias}, journal = {IEEE Internet of Things}, keywords = {t_journalmagazine}, month = {08}, number = 15, pages = {13974-13987}, title = {MVNOCoreSim: A Digital Twin for Virtualized IoT-centric Mobile Core Networks}, volume = 10, year = 2023 }
• 
  Serverless Computing: What It Is, and What It Is Not?. S. Kounev; N. Herbst; C. L. Abad; A. Iosup; I. Foster; P. Shenoy; O. Rana; A. A. Chien; in Commun. ACM (2023). 66(9) 80–92.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Dispelling the confusion around serverless computing by capturing its essential and conceptual characteristics.

  @article{10.1145/3587249, abstract = {Dispelling the confusion around serverless computing by capturing its essential and conceptual characteristics.}, address = {New York, NY, USA}, author = {Kounev, Samuel and Herbst, Nikolas and Abad, Cristina L. and Iosup, Alexandru and Foster, Ian and Shenoy, Prashant and Rana, Omer and Chien, Andrew A.}, journal = {Commun. ACM}, keywords = {t_journalmagazine}, month = {08}, number = 9, pages = {80–92}, publisher = {Association for Computing Machinery}, title = {Serverless Computing: What It Is, and What It Is Not?}, volume = 66, year = 2023 }
• 
  De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem. T. Prantl; L. Horn; S. Engel; L. Iffländer; L. Beierlieb; C. Krupitzer; A. Bauer; M. Sakarvadia; I. Foster; S. Kounev; in International Journal of Information Security (2023).
  • [ BibTeX ]
  • [ DOI ]
  @article{PrHoEnIfBeKrBaSaFoKo_IJoIS_SGC_Pareto, author = {Prantl, Thomas and Horn, Lukas and Engel, Simon and Iffl\"ander, Lukas and Beierlieb, Lukas and Krupitzer, Christian and Bauer, Andr\'e and Sakarvadia, Mansi and Foster, Ian and Kounev, Samuel}, journal = {International Journal of Information Security}, keywords = {t_journalmagazine}, month = 11, title = {De Bello Homomorphico: Investigation of the extensibility of the OpenFHE library with basic mathematical functions by means of common approaches using the example of the CKKS cryptosystem}, year = 2023 }
• 
  Email Discovery in Modern Applications. A. Pastukh; Thesis; Bachelor Thesis. (2023, July).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Email Discovery and Contact Discovery are processes that allow a user to find out which contacts from his phone book are using the same service. In the first case, the email address is used as an identifier, and in the second, the phone number. However, these processes come with privacy vulnerabilities that need to be addressed. One significant vulnerability is the potential for uploading all contacts and data of individuals who have not explicitly given permission for their information to be shared. When a user grants access to their phone book or contact list to a service, the people listed in their contacts have not necessarily consented to their data being uploaded to the server. This problem opens up the possibility of unauthorized access to sensitive data by third parties or even employees of the service provider. Data leaks and server attacks are also potential risks. Additionally, enumeration and crawling attacks pose threats by exploiting the ability to access data from a user’s phone book during the Email or Contact Discovery process. The danger of crawling attacks lies in the fact that they can be carried out without requiring special equipment or in-depth knowledge of the process for each specific service. As shown in the previous work, almost all modern instant messengers that use Contact Discovery are vulnerable to such crawling attacks. The purpose of our work is to study the possibility of carrying out crawling attacks on services that use an email address as an identifier, i.e., applications using Email Discovery. We consider instant messengers, job search applications, as well as gaming platforms and other applications. Our work shows that there are three applications using Email Discovery that turned out to be vulnerable to such crawling attacks: Xing, LinkedIn, and Google Contacts. To do this, we generate millions of email addresses that are then uploaded to the services to get information about whether the user is registered and to discover what data we can get with this information. The attacks carried out manually on the LinkedIn and Xing services proved to be ineffective primarily due to the implementation of Rate Limits. These limits restrict the number of requests allowed within a specific time frame, thereby hindering the success of our attacks. Additionally, the inclusion of user IP Address verification further contributed to the inability to extract user data successfully. As a result, the outcome of our attacks yielded insufficient results in terms of accessing and retrieving user data. The attack on the Google Contacts application shows the most devastating results. We develop a crawler, which allowed us to carry out a major crawling attack on this application. As a result, we are able to check 3046656 generated contacts. We also find that there are no Rate Limits enforced by Google to prevent such attacks. Thus, we show that, using a regular user PC, it is possible to carry out a crawling attack and find out such personal data of users as first name, last name, photo, links to other social networks, as well as online status. Furthermore, we analyze existing methods for protecting the Email Discovery process and propose privacy enhancements for the Google Contacts service.

  @mastersthesis{pastukh2023email, abstract = {Email Discovery and Contact Discovery are processes that allow a user to find out which contacts from his phone book are using the same service. In the first case, the email address is used as an identifier, and in the second, the phone number. However, these processes come with privacy vulnerabilities that need to be addressed. One significant vulnerability is the potential for uploading all contacts and data of individuals who have not explicitly given permission for their information to be shared. When a user grants access to their phone book or contact list to a service, the people listed in their contacts have not necessarily consented to their data being uploaded to the server. This problem opens up the possibility of unauthorized access to sensitive data by third parties or even employees of the service provider. Data leaks and server attacks are also potential risks. Additionally, enumeration and crawling attacks pose threats by exploiting the ability to access data from a user’s phone book during the Email or Contact Discovery process. The danger of crawling attacks lies in the fact that they can be carried out without requiring special equipment or in-depth knowledge of the process for each specific service. As shown in the previous work, almost all modern instant messengers that use Contact Discovery are vulnerable to such crawling attacks. The purpose of our work is to study the possibility of carrying out crawling attacks on services that use an email address as an identifier, i.e., applications using Email Discovery. We consider instant messengers, job search applications, as well as gaming platforms and other applications. Our work shows that there are three applications using Email Discovery that turned out to be vulnerable to such crawling attacks: Xing, LinkedIn, and Google Contacts. To do this, we generate millions of email addresses that are then uploaded to the services to get information about whether the user is registered and to discover what data we can get with this information. The attacks carried out manually on the LinkedIn and Xing services proved to be ineffective primarily due to the implementation of Rate Limits. These limits restrict the number of requests allowed within a specific time frame, thereby hindering the success of our attacks. Additionally, the inclusion of user IP Address verification further contributed to the inability to extract user data successfully. As a result, the outcome of our attacks yielded insufficient results in terms of accessing and retrieving user data. The attack on the Google Contacts application shows the most devastating results. We develop a crawler, which allowed us to carry out a major crawling attack on this application. As a result, we are able to check 3046656 generated contacts. We also find that there are no Rate Limits enforced by Google to prevent such attacks. Thus, we show that, using a regular user PC, it is possible to carry out a crawling attack and find out such personal data of users as first name, last name, photo, links to other social networks, as well as online status. Furthermore, we analyze existing methods for protecting the Email Discovery process and propose privacy enhancements for the Google Contacts service.}, author = {Pastukh, Andrii}, keywords = {sssgroup}, month = {07}, publisher = {Bachelor Thesis}, school = {University of Würzburg}, title = {Email Discovery in Modern Applications}, type = {Bachelor Thesis}, year = 2023 }
• 
  Penetration Testing of the eSano Platform with the focus on Patient and eCoach components. A. Nolte; Thesis; Bachelor Thesis. (2023, July).
  • [ BibTeX ]
  • [ Download ]
  @mastersthesis{nolte2023penetration, author = {Nolte, Alexander}, keywords = {sssgroup}, month = {07}, publisher = {Bachelor Thesis}, school = {University of Würzburg}, title = {Penetration Testing of the eSano Platform with the focus on Patient and eCoach components}, type = {Bachelor Thesis}, year = 2023 }
• 
  Adversarial Training in Federated Learning using Constrained Optimization Methods. J. König; Thesis; University of Würzburg. (2023, May).
  • [ BibTeX ]
  @mastersthesis{jan2023adversarial, author = {König, Jan}, keywords = {sss-group}, month = {05}, school = {University of Würzburg}, title = {Adversarial Training in Federated Learning using Constrained Optimization Methods}, type = {Master Thesis}, year = 2023 }
• 
  Self-Aware Optimization of Adaptation Planning Strategies. V. Lesch; M. Hadry; C. Krupitzer; S. Kounev; in ACM Transactions on Autonomous and Adaptive Systems (2023). 18(3)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  In today’s world, circumstances, processes, and requirements for software systems are becoming increasingly complex. In order to operate properly in such dynamic environments, software systems must adapt to these changes, which has led to the research area of Self-Adaptive Systems (SAS). Platooning is one example of adaptive systems in Intelligent Transportation Systems, which is the ability of vehicles to travel with close inter-vehicle distances. This technology leads to an increase in road throughput and safety, which directly addresses the increased infrastructure needs due to increased traffic on the roads. However, the No-Free-Lunch theorem states that the performance of one adaptation planning strategy is not necessarily transferable to other problems. Moreover, especially in the field of SAS, the selection of the most appropriate strategy depends on the current situation of the system. In this paper, we address the problem of self-aware optimization of adaptation planning strategies by designing a framework that includes situation detection, strategy selection, and parameter optimization of the selected strategies. We apply our approach on the case study platooning coordination and evaluate the performance of the proposed framework.

  @article{lesch2022selfaware, abstract = {In today’s world, circumstances, processes, and requirements for software systems are becoming increasingly complex. In order to operate properly in such dynamic environments, software systems must adapt to these changes, which has led to the research area of Self-Adaptive Systems (SAS). Platooning is one example of adaptive systems in Intelligent Transportation Systems, which is the ability of vehicles to travel with close inter-vehicle distances. This technology leads to an increase in road throughput and safety, which directly addresses the increased infrastructure needs due to increased traffic on the roads. However, the No-Free-Lunch theorem states that the performance of one adaptation planning strategy is not necessarily transferable to other problems. Moreover, especially in the field of SAS, the selection of the most appropriate strategy depends on the current situation of the system. In this paper, we address the problem of self-aware optimization of adaptation planning strategies by designing a framework that includes situation detection, strategy selection, and parameter optimization of the selected strategies. We apply our approach on the case study platooning coordination and evaluate the performance of the proposed framework.}, author = {Lesch, Veronika and Hadry, Marius and Krupitzer, Christian and Kounev, Samuel}, journal = {ACM Transactions on Autonomous and Adaptive Systems}, keywords = {self-awareness}, month = {09}, number = 3, title = {Self-Aware Optimization of Adaptation Planning Strategies}, volume = 18, year = 2023 }
• 
  SPEC Research Group Newsletter, Issue 12. S. Kounev; A. van Hoorn; M. Straesser; A. Bauer; (2023, April).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-12, author = {Kounev, Samuel and van Hoorn, Andre and Straesser, Martin and Bauer, Andr{é}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {SPEC Research Group Newsletter, Issue 12}, year = 2023 }
• 
  A Trace-driven Performance Evaluation of Hash-based Task Placement Algorithms for Cache-enabled Serverless Computing. S. Talluri; N. Herbst; C. Abad; A. Trivedi; A. Iosup; in 2023 ACM International Conference on Computing Frontiers (CF ’23) (2023).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{2023-cf-hash-task-placement, author = {Talluri, Sacheendra and Herbst, Nikolas and Abad, Cristina and Trivedi, Animesh and Iosup, Alexandru}, booktitle = {2023 ACM International Conference on Computing Frontiers (CF '23)}, keywords = {sys:relevantfor:se-group}, title = {A Trace-driven Performance Evaluation of Hash-based Task Placement Algorithms for Cache-enabled Serverless Computing}, year = 2023 }
• 
  Optimizing storage assignment, order picking, and their interaction in mezzanine warehouses. V. Lesch; P. B. Müller; M. Krämer; M. Hadry; S. Kounev; C. Krupitzer; in Applied intelligence (2023). 25.
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  @article{lesch2023optimizing, author = {Lesch, Veronika and Müller, Patrick B.M. and Krämer, Moritz and Hadry, Marius and Kounev, Samuel and Krupitzer, Christian}, journal = {Applied intelligence}, keywords = {t_journalmagazine}, month = {02}, pages = 25, title = {Optimizing storage assignment, order picking, and their interaction in mezzanine warehouses}, year = 2023 }
• 
  A literature review of IoT and CPS—What they are, and what they are not. V. Lesch; M. Züfle; A. Bauer; L. Iffländer; C. Krupitzer; S. Kounev; in Journal of Systems and Software (2023). 111631.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Today, we are confronted with many concepts such as Cyber–Physical-Systems (CPS), Internet of Things (IoT), Industry 4.0, Industrial Internet, Ubiquitous Computing, Pervasive Computing and many more. Some researchers use all of these terms interchangeably, while others interpret them in ways that contradict each other. The inconsistent and interchangeable usage of these terms creates the impression that authors abuse them as buzzwords to attract attention. Hence, the question arises: Is the existence of all these terms justified? In this paper, we first look at the origin of the terms. Then, we focus on Internet of Things (IoT) and Cyber-Physical-Systems (CPS) as those terms are more often used as the others and further often seen as underlying technologies. We presentthe results of a literature review, including academic, industry and gray literature, with the objective to identify and discuss several clusters of similar statements on both terms. Building on this, we present definitions for IoT and CPS that reflect the core intuition of the terms as found in the literature review while providing a clear demarcation of the two terms. Then, we illustrate the applicability of our findings on several use cases. Finally, we discuss the relation to the other topics closely related to adaptive systems, namely Industry 4.0, Industrial Internet, Ubiquitous Computing, and Pervasive Computing.

  @article{lesch2023literature, abstract = {Today, we are confronted with many concepts such as Cyber–Physical-Systems (CPS), Internet of Things (IoT), Industry 4.0, Industrial Internet, Ubiquitous Computing, Pervasive Computing and many more. Some researchers use all of these terms interchangeably, while others interpret them in ways that contradict each other. The inconsistent and interchangeable usage of these terms creates the impression that authors abuse them as buzzwords to attract attention. Hence, the question arises: Is the existence of all these terms justified? In this paper, we first look at the origin of the terms. Then, we focus on Internet of Things (IoT) and Cyber-Physical-Systems (CPS) as those terms are more often used as the others and further often seen as underlying technologies. We presentthe results of a literature review, including academic, industry and gray literature, with the objective to identify and discuss several clusters of similar statements on both terms. Building on this, we present definitions for IoT and CPS that reflect the core intuition of the terms as found in the literature review while providing a clear demarcation of the two terms. Then, we illustrate the applicability of our findings on several use cases. Finally, we discuss the relation to the other topics closely related to adaptive systems, namely Industry 4.0, Industrial Internet, Ubiquitous Computing, and Pervasive Computing.}, author = {Lesch, Veronika and Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Iffl{\"a}nder, Lukas and Krupitzer, Christian and Kounev, Samuel}, journal = {Journal of Systems and Software}, keywords = {t_journalmagazine}, pages = 111631, publisher = {Elsevier}, title = {A literature review of IoT and CPS—What they are, and what they are not}, year = 2023 }
• 
  Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field. A. Bauer; M. Leznik; M. Stenger; R. Leppich; N. Herbst; S. Kounev; I. Foster; (2023).
  • [ BibTeX ]
  • [ Download ]
  @misc{bauer2023telescope, author = {Bauer, André and Leznik, Mark and Stenger, Michael and Leppich, Robert and Herbst, Nikolas and Kounev, Samuel and Foster, Ian}, keywords = {t_journalmagazine}, title = {Telescope: An Automated Hybrid Forecasting Approach on a Level-Playing Field}, year = 2023 }
• 
  Security of NVMe Offloaded Data in Large-Scale Machine Learning. T. Krauß; G. Raphael; A. Dmitrienko; in To appear soon at the European Symposium on Research in Computer Security (ESORICS) (2023).
  • [ BibTeX ]
  • [ URL ]
  @article{krauss2024security, author = {Krauß, Torsten and Raphael, Götz and Dmitrienko, Alexandra}, journal = {To appear soon at the European Symposium on Research in Computer Security (ESORICS)}, keywords = {Private_AI}, title = {Security of NVMe Offloaded Data in Large-Scale Machine Learning}, year = 2023 }
• 
  Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations T. Krauß; A. Dmitrienko; (2023).
  • [ BibTeX ]
  • [ URL ]
  @techreport{krauss2023avoid, author = {Krauß, Torsten and Dmitrienko, Alexandra}, keywords = {Machine}, title = {Avoid Adversarial Adaption in Federated Learning by Multi-Metric Investigations}, type = {arXiv:2306.03600}, year = 2023 }
• 
  Association between Postoperative Atrial Fibrillation Recurrences and Interatrial Block by the Time of Open-Heart Surgery. S. Leiler; J. Kalisnik; R. Bernik; A. Bauer; V. Günzler; P. Sluet; S. Kounev; T. Fischlein; in The Thoracic and Cardiovascular Surgeon (2023). 71(S 01) DGTHG-V130.
  • [ BibTeX ]
  @article{leiler2023association, author = {Leiler, S and Kalisnik, JM and Bernik, R and Bauer, A and G{\"u}nzler, V and Sluet, P and Kounev, S and Fischlein, T}, journal = {The Thoracic and Cardiovascular Surgeon}, keywords = {t_journalmagazine}, number = {S 01}, pages = {DGTHG–V130}, publisher = {Georg Thieme Verlag KG}, title = {Association between Postoperative Atrial Fibrillation Recurrences and Interatrial Block by the Time of Open-Heart Surgery}, volume = 71, year = 2023 }
• 
  AuthentiSense: A Scalable Behavioral Biometrics Authentication Scheme using Few-Shot Learning for Mobile Platforms. H. Fereidooni; J. König; P. Rieger; M. Chilese; M. Finke; B. Goekbakan; A. Dmitrienko; A.-R. Sadeghi; in To appear at the Network and Distributed System Security Symposium (NDSS) (2023).
  • [ BibTeX ]
  @article{hossein2023authentisense, author = {Fereidooni, Hossein and König, Jan and Rieger, Philipp and Chilese, Markus and Finke, Moritz and Goekbakan, Bora and Dmitrienko, Alexandra and Sadeghi, Ahmad-Reza}, journal = {To appear at the Network and Distributed System Security Symposium (NDSS)}, keywords = {sss-group}, title = {AuthentiSense: A Scalable Behavioral Biometrics Authentication Scheme using Few-Shot Learning for Mobile Platforms}, year = 2023 }
• 
  Smarter Contracts: Detecting Vulnerabilities in Smart Contracts with Deep Transfer Learning. C. Sendner; H. Chen; H. Fereidooni; L. Petzi; J. König; J. Stang; A. Dmitrienko; A.-R. Sadeghi; F. Koushanfar; in To appear at the Network and Distributed System Security Symposium (NDSS) (2023).
  • [ BibTeX ]
  @article{sendner2023smarter, author = {Sendner, Christoph and Chen, Huili and Fereidooni, Hossein and Petzi, Lukas and König, Jan and Stang, Jasper and Dmitrienko, Alexandra and Sadeghi, Ahmad-Reza and Koushanfar, Farinaz}, journal = {To appear at the Network and Distributed System Security Symposium (NDSS)}, keywords = {sss-group}, title = {Smarter Contracts: Detecting Vulnerabilities in Smart Contracts with Deep Transfer Learning}, year = 2023 }
• 
  Gamification of Ethical Hacking Lab. J. Schraud; Thesis; Bachelor Thesis. (2023, February).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Education is an important topic. Increasing the effectiveness and efficiency of conveying knowledge is interesting for schools of all kinds, in the professional world, but also in the private sector. In this paper, we deal with this highly important topic. Thus, we implement a promising approach for improving education called gamification, for the module Ethical Hacking Lab (EHL) of the Secure Software Systems Group of the Julius-Maximilians- University of Wuerzburg. Summarized into one sentence, in this paper we improve the EHL through gamification in the form of a so-called Capture The Flag (CTF) project. Specifically, on the one hand, we develop a CTF framework to provide and administer exercises. The framework also allows the integration of virtual laboratory environments. Furthermore, it contains statistics and various game elements to promote the game char- acteristics and to motivate the participants. On the other hand, we develop virtual laboratory environments for the binary exploitation exercises of the EHL. Hereby, we aim to further motivate and engage the participants by enabling an actual execution of their developed exploits against a virtual target. In addition, our laboratory environments relieve the participants from tedious setup tasks, as well as the tutors of the EHL from the manual correction of the submitted solutions. Furthermore, we integrate all other exercises of the EHL into our CTF framework. We implement this by extracting and integrating various questions about the exercises. By developing and executing detailed test plans, we ultimately ensure proper functionality of the CTF project.

  @mastersthesis{schraud2023gamification, abstract = {Education is an important topic. Increasing the effectiveness and efficiency of conveying knowledge is interesting for schools of all kinds, in the professional world, but also in the private sector. In this paper, we deal with this highly important topic. Thus, we implement a promising approach for improving education called gamification, for the module Ethical Hacking Lab (EHL) of the Secure Software Systems Group of the Julius-Maximilians- University of Wuerzburg. Summarized into one sentence, in this paper we improve the EHL through gamification in the form of a so-called Capture The Flag (CTF) project. Specifically, on the one hand, we develop a CTF framework to provide and administer exercises. The framework also allows the integration of virtual laboratory environments. Furthermore, it contains statistics and various game elements to promote the game char- acteristics and to motivate the participants. On the other hand, we develop virtual laboratory environments for the binary exploitation exercises of the EHL. Hereby, we aim to further motivate and engage the participants by enabling an actual execution of their developed exploits against a virtual target. In addition, our laboratory environments relieve the participants from tedious setup tasks, as well as the tutors of the EHL from the manual correction of the submitted solutions. Furthermore, we integrate all other exercises of the EHL into our CTF framework. We implement this by extracting and integrating various questions about the exercises. By developing and executing detailed test plans, we ultimately ensure proper functionality of the CTF project.}, author = {Schraud, Johannes}, keywords = {sssgroup}, month = {02}, publisher = {Bachelor Thesis}, school = {University of Würzburg}, title = {Gamification of Ethical Hacking Lab}, type = {Bachelor Thesis}, year = 2023 }

• 
  ICPE ’22: ACM/SPEC International Conference on Performance Engineering, Bejing, China, April 9 - 13, 2022, Companion Volume D. Feng; S. Becker; N. Herbst; P. Leitner; (D. Feng; S. Becker; N. Herbst; P. Leitner, eds.) (2022). ACM.
  • [ BibTeX ]
  • [ URL ]
  @proceedings{DBLP:conf/wosp/2022c, author = {Feng, Dan and Becker, Steffen and Herbst, Nikolas and Leitner, Philipp}, editor = {Feng, Dan and Becker, Steffen and Herbst, Nikolas and Leitner, Philipp}, keywords = {t_proceedings}, publisher = {{ACM}}, title = {{ICPE} '22: {ACM/SPEC} International Conference on Performance Engineering, Bejing, China, April 9 - 13, 2022, Companion Volume}, year = 2022 }
• 
  CortexVR: Immersive analysis and training of cognitive executive functions of soccer players using virtual reality and machine learning. C. Krupitzer; J. Naber; J.-P. Stauffert; J. Mayer; J. Spielmann; P. Ehmann; N. Boci; M. Bürkle; A. Ho; C. Komorek; F. Heinickel; S. Kounev; C. Becker; M. E. Latoschik; in Frontiers in Psychology (2022). 5139.
  • [ BibTeX ]
  • [ Download ]
  @article{krupitzer2022cortexvr, author = {Krupitzer, Christian and Naber, Jens and Stauffert, Jan-Philipp and Mayer, Jan and Spielmann, Jan and Ehmann, Paul and Boci, Noel and B{\"u}rkle, Maurice and Ho, Andr{\'e} and Komorek, Clemens and Heinickel, Felix and Kounev, Samuel and Becker, Christian and Latoschik, Marc Erich}, journal = {Frontiers in Psychology}, keywords = {t_journalmagazine}, pages = 5139, publisher = {Frontiers}, title = {CortexVR: Immersive analysis and training of cognitive executive functions of soccer players using virtual reality and machine learning}, year = 2022 }
• 
  SCRAPS: Scalable Collective Remote Attestation for Pub-Sub IoT Networks with Untrusted Proxy Verifier L. Petzi; A. E. Ben Yahya; A. Dmitrienko; G. Tsudik; T. Prantl; S. Kounev; (2022). (Vol. USENIX Security ’22)

  Coming Soon
  • [ Abstract ]
  • [ BibTeX ]
  Remote Attestation (RA) is a basic security mechanism that detects malicious presence on various types of computing components, e.g., IoT devices. In a typical IoT setting, RA involves a trusted Verifier that sends a challenge to an untrusted remote Prover, which must in turn reply with a fresh and authentic evidence of being in a trustworthy state. However,most current RA schemes assume a central Verifier, which rep-resents a single point of failure. This feature is problematic when mutually suspicious stakeholders are involved. Furthermore, scalability issues arise as the number of IoT devices (Provers) grows. Although some RA schemes allow peer Provers to act as Verifiers, they involve unrealistic (for IoT devices) requirements, such as time synchronization and synchronous communication. Moreover, they incur heavy memory, computation,and communication burdens, while not considering sleeping or otherwise disconnected devices. Motivated by the need to address these limitations, we construct Scalable Collective Remote Attestation for Pub-Sub (SCRAPS), a novel collective RA scheme. It achieves scalability by outsourcing Verifier duties to a smart contract and mitigates DoS attacks against both Provers and Verifiers. It also removes the need for synchronous communication. Furthermore,RA evidence in SCRAPS is publicly verifiable, which significantly reduces the number of attestation evidence computations, thus lowering Prover burden. We report on a prototype of SCRAPS over Hyperledger Sawtooth (a blockchain geared for IoT use-cases) and evaluate its performance, scalability, and security aspects.

  @proceedings{petzi2022scraps, abstract = {Remote Attestation (RA) is a basic security mechanism that detects malicious presence on various types of computing components, e.g., IoT devices. In a typical IoT setting, RA involves a trusted Verifier that sends a challenge to an untrusted remote Prover, which must in turn reply with a fresh and authentic evidence of being in a trustworthy state. However,most current RA schemes assume a central Verifier, which rep-resents a single point of failure. This feature is problematic when mutually suspicious stakeholders are involved. Furthermore, scalability issues arise as the number of IoT devices (Provers) grows. Although some RA schemes allow peer Provers to act as Verifiers, they involve unrealistic (for IoT devices) requirements, such as time synchronization and synchronous communication. Moreover, they incur heavy memory, computation,and communication burdens, while not considering sleeping or otherwise disconnected devices. Motivated by the need to address these limitations, we construct Scalable Collective Remote Attestation for Pub-Sub (SCRAPS), a novel collective RA scheme. It achieves scalability by outsourcing Verifier duties to a smart contract and mitigates DoS attacks against both Provers and Verifiers. It also removes the need for synchronous communication. Furthermore,RA evidence in SCRAPS is publicly verifiable, which significantly reduces the number of attestation evidence computations, thus lowering Prover burden. We report on a prototype of SCRAPS over Hyperledger Sawtooth (a blockchain geared for IoT use-cases) and evaluate its performance, scalability, and security aspects.}, author = {Petzi, Lukas and Ben Yahya, Ala Eddine and Dmitrienko, Alexandra and Tsudik, Gene and Prantl, Thomas and Kounev, Samuel}, keywords = {sss-group}, note = {Coming Soon}, title = {SCRAPS: Scalable Collective Remote Attestation for Pub-Sub IoT Networks with Untrusted Proxy Verifier}, volume = {USENIX Security '22}, year = 2022 }
• 
  Retrospektive Analyse ausgewählter Belastungs-und Beanspruchungsparameter einer professionellen Fußball-mannschaft vor und während coronabedingter Kontakt-beschränkungen in der Spielsaison 2019/20. P. Kunz; P. Düking; R. Leppich; J. Lachmann; B. Sperlich; in Leipziger Sportwissenschaftliche Beiträge Jahrgang 63 (2022 Heft 1 (2022). 75.
  • [ BibTeX ]
  @article{kunz2022retrospektive, author = {Kunz, Philipp and D{\"u}king, Peter and Leppich, Robert and Lachmann, Jonas and Sperlich, Billy}, journal = {Leipziger Sportwissenschaftliche Beitr{\"a}ge Jahrgang 63 (2022 Heft 1}, keywords = {t_interdisciplinary}, pages = 75, publisher = {Lehmanns Media}, title = {Retrospektive Analyse ausgew{\"a}hlter Belastungs-und Beanspruchungsparameter einer professionellen Fu{\ss}ball-mannschaft vor und w{\"a}hrend coronabedingter Kontakt-beschr{\"a}nkungen in der Spielsaison 2019/20}, year = 2022 }
• 
  Leveraging Kubernetes Source Code for Performance Simulation M. Straesser; P. Haas; (2022).
  • [ BibTeX ]
  • [ URL ]
  @presentation{straesser2022leveraging, author = {Straesser, Martin and Haas, Patrick}, keywords = {container_orchestration}, title = {Leveraging Kubernetes Source Code for Performance Simulation}, year = 2022 }
• 
  Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes. T. Prantl; T. Zeck; L. Iffländer; L. Beierlieb; A. Dmitrenko; C. Krupitzer; S. Kounev; in 2022 5th Conference on Cloud and Internet of Things (CIoT) (2022).
  • [ BibTeX ]
  @inproceedings{PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark, author = {Prantl, Thomas and Zeck, Timo and Iffländer, Lukas and Beierlieb, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2022 5th Conference on Cloud and Internet of Things (CIoT)}, keywords = {descartes}, organization = {IEEE}, title = {Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes}, year = 2022 }
• 
  Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity. M. Leznik; J. Grohmann; N. Kliche; A. Bauer; D. Seybold; S. Eismann; S. Kounev; J. Domaschka; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 89–96.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.

  @inproceedings{leznik2022dissimilar, abstract = {Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.}, address = {New York, NY, USA}, author = {Leznik, Mark and Grohmann, Johannes and Kliche, Nina and Bauer, Andr\'{e} and Seybold, Daniel and Eismann, Simon and Kounev, Samuel and Domaschka, J\"{o}rg}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {analysis}, pages = {89–96}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity}, year = 2022 }
• 
  Contact Discovery in Mobile Messengers: Low-cost Attacks, Quantitative Analyses, and Efficient Mitigations. C. Hagen; C. Weinert; C. Sendner; A. Dmitrienko; T. Schneider; in ACM Transactions on Privacy and Security (2022).

  (to appear)
  • [ BibTeX ]
  @article{hagen2022contact, author = {Hagen, Christoph and Weinert, Christian and Sendner, Christoph and Dmitrienko, Alexandra and Schneider, Thomas}, journal = {ACM Transactions on Privacy and Security}, keywords = {sss-group}, note = {(to appear)}, title = {Contact Discovery in Mobile Messengers: Low-cost Attacks, Quantitative Analyses, and Efficient Mitigations}, year = 2022 }
• 
  A data-driven Sensor Model for LIDAR Range Measurements used for Mobile Robot Navigation. F. Spiess; N. Strobel; T. Kaupp; S. Kounev; in Proceedings of the IEEE IRC 2022 Conference (2022).
  • [ BibTeX ]
  @conference{spiess2022datadriven, author = {Spiess, Florian and Strobel, Norbert and Kaupp, Tobias and Kounev, Samuel}, booktitle = {Proceedings of the IEEE IRC 2022 Conference}, keywords = {t_interdisciplinary}, title = {A data-driven Sensor Model for LIDAR Range Measurements used for Mobile Robot Navigation}, year = 2022 }
• 
  The State of Serverless Applications: Collection, Characterization, and Community Consensus. S. Eismann; J. Scheuner; E. van Eyk; M. Schwinger; J. Grohmann; N. Herbst; C. Abad; A. Iosup; in Transactions on Software Engineering (2022). 48(10) 4152–4166.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Over the last five years, all major cloud platform providers have increased their serverless offerings. Many early adopters report significant benefits for serverless-based over traditional applications, and many companies are considering moving to serverless themselves. However, currently there exist only few, scattered, and sometimes even conflicting reports on when serverless applications are well suited and what the best practices for their implementation are. We address this problem in the present study about the state of serverless applications. We collect descriptions of 89 serverless applications from open-source projects, academic literature, industrial literature, and domain-specific feedback. We analyze 16 characteristics that describe why and when successful adopters are using serverless applications, and how they are building them. We further compare the results of our characterization study to 10 existing, mostly industrial, studies and datasets; this allows us to identify points of consensus across multiple studies, investigate points of disagreement, and overall confirm the validity of our results. The results of this study can help managers to decide if they should adopt serverless technology, engineers to learn about current practices of building serverless applications, and researchers and platform providers to better understand the current landscape of serverless applications.

  @article{eismann2021state, abstract = {Over the last five years, all major cloud platform providers have increased their serverless offerings. Many early adopters report significant benefits for serverless-based over traditional applications, and many companies are considering moving to serverless themselves. However, currently there exist only few, scattered, and sometimes even conflicting reports on when serverless applications are well suited and what the best practices for their implementation are. We address this problem in the present study about the state of serverless applications. We collect descriptions of 89 serverless applications from open-source projects, academic literature, industrial literature, and domain-specific feedback. We analyze 16 characteristics that describe why and when successful adopters are using serverless applications, and how they are building them. We further compare the results of our characterization study to 10 existing, mostly industrial, studies and datasets; this allows us to identify points of consensus across multiple studies, investigate points of disagreement, and overall confirm the validity of our results. The results of this study can help managers to decide if they should adopt serverless technology, engineers to learn about current practices of building serverless applications, and researchers and platform providers to better understand the current landscape of serverless applications.}, author = {Eismann, Simon and Scheuner, Joel and van Eyk, Erwin and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina and Iosup, Alexandru}, journal = {Transactions on Software Engineering}, keywords = {t_journalmagazine}, number = 10, pages = {4152- 4166}, title = {The State of Serverless Applications: Collection, Characterization, and Community Consensus}, volume = 48, year = 2022 }
• 
  Close the Gate: Detecting Backdoored Models in Federated Learning based on Client-Side Deep Layer Output Analysis. P. Rieger; T. Krauß; M. Miettinen; A. Dmitrienko; A.-R. Sadeghi; in ArXiv | arXiv:2210.07714 (2022).
  • [ BibTeX ]
  • [ URL ]
  @article{https://doi.org/10.48550/arxiv.2210.07714, author = {Rieger, Phillip and Krauß, Torsten and Miettinen, Markus and Dmitrienko, Alexandra and Sadeghi, Ahmad-Reza}, journal = {ArXiv | arXiv:2210.07714}, keywords = {Private_AI}, publisher = {arXiv}, title = {Close the Gate: Detecting Backdoored Models in Federated Learning based on Client-Side Deep Layer Output Analysis}, year = 2022 }
• 
  Ransomware Detection in Databases through Dynamic Analysis of Query Sequences. C. Sendner; L. Iffländer; S. Schindler; M. Jobst; A. Dmitrienko; S. Kounev; in 2022 IEEE Conference on Communications and Network Security (CNS) (2022).
  • [ BibTeX ]
  @conference{sendner2022ransomware, author = {Sendner, Christoph and Iffländer, Lukas and Schindler, Sebastian and Jobst, Michael and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {2022 IEEE Conference on Communications and Network Security (CNS)}, keywords = {sss-group}, title = {Ransomware Detection in Databases through Dynamic Analysis of Query Sequences}, year = 2022 }
• 
  A Survey on Secure Group Communication Schemes with Focus on IoT Communication. T. Prantl; T. Zeck; A. Bauer; P. Ten; D. Prantl; A. E. Ben Yahya; L. Iffländer; A. Dmitrienko; C. Krupitzer; S. Kounev; in IEEE Access (2022).

  Accepted but not yet published
  • [ BibTeX ]
  @article{PrZeBaTePrYaIfDmKrKo_IEEEAccess_SGCSurvey, author = {Prantl, Thomas and Zeck, Timo and Bauer, André and Ten, Peter and Prantl, Dominik and Ben Yahya, Ala Eddine and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, journal = {IEEE Access}, keywords = {t_journalmagazine}, note = {Accepted but not yet published}, publisher = {IEEE}, title = {A Survey on Secure Group Communication Schemes with Focus on IoT Communication}, year = 2022 }
• 
  Retrospective Analysis of Training Intensity Distribution Based on Race Pace Versus Physiological Benchmarks in Highly Trained Sprint Kayakers. M. Matzka; R. Leppich; B. Sperlich; C. Zinner; in Sports Medicine-Open (2022). 8(1) 1–12.
  • [ BibTeX ]
  @article{matzka2022retrospective, author = {Matzka, Manuel and Leppich, Robert and Sperlich, Billy and Zinner, Christoph}, journal = {Sports Medicine-Open}, keywords = {t_journalmagazine}, number = 1, pages = {1–12}, publisher = {SpringerOpen}, title = {Retrospective Analysis of Training Intensity Distribution Based on Race Pace Versus Physiological Benchmarks in Highly Trained Sprint Kayakers}, volume = 8, year = 2022 }
• 
  Accuracy and Systematic Biases of Heart Rate Measurements by Consumer-Grade Fitness Trackers in Postoperative Patients: Prospective Clinical Trial. P. Helmer; S. Hottenrott; P. Rodemers; R. Leppich; M. Helwich; R. Pryss; P. Kranke; P. Meybohm; B. E. Winkler; M. Sammeth; in Journal of Medical Internet Research (2022). 24(12) e42359.
  • [ BibTeX ]
  @article{helmer2022accuracy, author = {Helmer, Philipp and Hottenrott, Sebastian and Rodemers, Philipp and Leppich, Robert and Helwich, Maja and Pryss, R{\"u}diger and Kranke, Peter and Meybohm, Patrick and Winkler, Bernd E and Sammeth, Michael}, journal = {Journal of Medical Internet Research}, keywords = {t_journalmagazine}, number = 12, pages = {e42359}, publisher = {JMIR Publications Toronto, Canada}, title = {Accuracy and Systematic Biases of Heart Rate Measurements by Consumer-Grade Fitness Trackers in Postoperative Patients: Prospective Clinical Trial}, volume = 24, year = 2022 }
• 
  ICPE ’22: ACM/SPEC International Conference on Performance Engineering, Bejing, China, April 9 - 13, 2022 D. Feng; S. Becker; N. Herbst; P. Leitner; (D. Feng; S. Becker; N. Herbst; P. Leitner, eds.) (2022). ACM.
  • [ BibTeX ]
  • [ URL ]
  @proceedings{DBLP:conf/wosp/2022, author = {Feng, Dan and Becker, Steffen and Herbst, Nikolas and Leitner, Philipp}, editor = {Feng, Dan and Becker, Steffen and Herbst, Nikolas and Leitner, Philipp}, keywords = {t_proceedings}, publisher = {{ACM}}, title = {{ICPE} '22: {ACM/SPEC} International Conference on Performance Engineering, Bejing, China, April 9 - 13, 2022}, year = 2022 }
• 
  MiSim: A Simulator for Resilience Assessment of Microservice-based Architectures. S. Frank; L. Wagner; A. Hakamian; M. Straesser; A. van Hoorn; in 2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS) (2022). 1014–1025.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  Increased resilience compared to monolithic architectures is both one of the key promises of microservice-based architectures and a big challenge, e.g., due to the systems’ distributed nature. Resilience assessment through simulation requires fewer resources than the measurement-based techniques used in practice. However, there is no existing simulation approach that is suitable for a holistic resilience assessment of microservices comprised of (i) representative fault injections, (ii) common resilience mechanisms, and (iii) time-varying workloads. This paper presents MiSim — an extensible simulator for resilience assessment of microservice-based architectures. It overcomes the stated limitations of related work. MiSim fits resilience engineering practices by supporting scenario-based experiments and requiring only lightweight input models. We demonstrate how MiSim simulates (1) common resilience mechanisms — i.e., circuit breaker, connection limiter, retry, load balancer, and autoscaler — and (2) fault injections — i.e., instance/service killing and latency injections. In addition, we use TeaStore, a reference microservice-based architecture, aiming to reproduce scaling behavior from an experiment by using simulation. Our results show that MiSim allows for quantitative insights into microservice-based systems’ complex transient behavior by providing up to 25 metrics.

  @inproceedings{FrankWagnerHakamianStraesserVanHoorn2022MiSim, abstract = {Increased resilience compared to monolithic architectures is both one of the key promises of microservice-based architectures and a big challenge, e.g., due to the systems’ distributed nature. Resilience assessment through simulation requires fewer resources than the measurement-based techniques used in practice. However, there is no existing simulation approach that is suitable for a holistic resilience assessment of microservices comprised of (i) representative fault injections, (ii) common resilience mechanisms, and (iii) time-varying workloads. This paper presents MiSim — an extensible simulator for resilience assessment of microservice-based architectures. It overcomes the stated limitations of related work. MiSim fits resilience engineering practices by supporting scenario-based experiments and requiring only lightweight input models. We demonstrate how MiSim simulates (1) common resilience mechanisms — i.e., circuit breaker, connection limiter, retry, load balancer, and autoscaler — and (2) fault injections — i.e., instance/service killing and latency injections. In addition, we use TeaStore, a reference microservice-based architecture, aiming to reproduce scaling behavior from an experiment by using simulation. Our results show that MiSim allows for quantitative insights into microservice-based systems’ complex transient behavior by providing up to 25 metrics.}, author = {Frank, Sebastian and Wagner, Lion and Hakamian, Alireza and Straesser, Martin and van Hoorn, Andre}, booktitle = {2022 IEEE 22nd International Conference on Software Quality, Reliability and Security (QRS)}, keywords = {resilience}, pages = {1014-1025}, publisher = {IEEE}, title = {MiSim: A Simulator for Resilience Assessment of Microservice-based Architectures}, year = 2022 }
• 
  Artificial Intelligence-Based Early Detection Of Acute Kidney Injury After Cardiac Surgery. J. M. Kalisnik; A. Bauer; F. A. Vogt; F. J. Stickl; J. Zibert; M. Fittkau; T. Bertsch; S. Kounev; T. Fischlein; in European Journal of Cardio-Thoracic Surgery (2022).

  Joint first authorship; To be published soon
  • [ Abstract ]
  • [ BibTeX ]
  Objectives: This study aims to improve early detection of cardiac surgery-associated acute kidney injury using artificial intelligence-based algorithms. Methods: Data from consecutive patients undergoing cardiac surgery between 2008 and 2018 in our institution served as the source for artificial intelligence-based modeling. Cardiac surgery-associated acute kidney injury was defined according to the Kidney Disease Improving Global Outcomes criteria. Different machine learning algorithms were trained and validated to detect cardiac surgery-associated acute kidney injury within 12 hours after surgery. Demographic characteristics, comorbidities, preoperative cardiac status, intra- and postoperative variables including creatinine and hemoglobin values were retrieved for analysis. Results: From 7507 patients analyzed, 1699 patients (22.6 %) developed cardiac surgery-associated acute kidney injury. The ultimate detection model, 'Detect-A(K)I', recognizes cardiac surgery-associated acute kidney injury within 12 hours with an area under the curve of 88.0 %, sensitivity of 78.0 %, specificity of 78.9 %, and accuracy of 82.1 %. The optimal parameter set includes serial changes of creatinine and hemoglobin, operative emergency, bleeding-associated variables, cardiac ischemic time and cardiac function-associated variables, age, diuretics and active infection, chronic obstructive lung and peripheral vascular disease. Conclusion: The 'Detect-A(K)I' model successfully detects cardiac surgery-associated acute kidney injury within 12 hours after surgery with the best discriminatory characteristics reported so far.

  @article{kalisnik2022artificial, abstract = {Objectives: This study aims to improve early detection of cardiac surgery-associated acute kidney injury using artificial intelligence-based algorithms. Methods: Data from consecutive patients undergoing cardiac surgery between 2008 and 2018 in our institution served as the source for artificial intelligence-based modeling. Cardiac surgery-associated acute kidney injury was defined according to the Kidney Disease Improving Global Outcomes criteria. Different machine learning algorithms were trained and validated to detect cardiac surgery-associated acute kidney injury within 12 hours after surgery. Demographic characteristics, comorbidities, preoperative cardiac status, intra- and postoperative variables including creatinine and hemoglobin values were retrieved for analysis. Results: From 7507 patients analyzed, 1699 patients (22.6 %) developed cardiac surgery-associated acute kidney injury. The ultimate detection model, 'Detect-A(K)I', recognizes cardiac surgery-associated acute kidney injury within 12 hours with an area under the curve of 88.0 %, sensitivity of 78.0 %, specificity of 78.9 %, and accuracy of 82.1 %. The optimal parameter set includes serial changes of creatinine and hemoglobin, operative emergency, bleeding-associated variables, cardiac ischemic time and cardiac function-associated variables, age, diuretics and active infection, chronic obstructive lung and peripheral vascular disease. Conclusion: The 'Detect-A(K)I' model successfully detects cardiac surgery-associated acute kidney injury within 12 hours after surgery with the best discriminatory characteristics reported so far.}, author = {Kalisnik, Jurij Matija and Bauer, Andr{é} and Vogt, Ferdinand Aurel and Stickl, Franziska Josephine and Zibert, Janez and Fittkau, Matthias and Bertsch, Thomas and Kounev, Samuel and Fischlein, Theodor}, journal = {European Journal of Cardio-Thoracic Surgery}, keywords = {t_journalmagazine}, note = {Joint first authorship; To be published soon}, title = {Artificial Intelligence-Based Early Detection Of Acute Kidney Injury After Cardiac Surgery}, year = 2022 }
• 
  Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling. M. Straesser; J. Grohmann; J. von Kistowski; S. Eismann; A. Bauer; S. Kounev; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 105–115.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.

  @inproceedings{10.1145/3489525.3511680, abstract = {Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.}, address = {New York, NY, USA}, author = {Straesser, Martin and Grohmann, Johannes and von Kistowski, J\'{o}akim and Eismann, Simon and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {microservices}, pages = {105–115}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling}, year = 2022 }
• 
  Investigating the Predictability of QoS Metrics in Cellular Networks. S. Herrnleben; J. Grohmann; V. Lesch; T. Prantl; F. Metzger; T. Hoßfeld; S. Kounev; in 2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS) (2022). 1–10.
  • [ BibTeX ]
  @inproceedings{HeGrLePrMeHoKo-2022-IWQoS-QoS-Prediction-Cellular-Networks, author = {Herrnleben, Stefan and Grohmann, Johannes and Lesch, Veronika and Prantl, Thomas and Metzger, Florian and Hoßfeld, Tobias and Kounev, Samuel}, booktitle = {2022 IEEE/ACM 30th International Symposium on Quality of Service (IWQoS)}, keywords = {descartes}, pages = {1-10}, title = {{Investigating the Predictability of QoS Metrics in Cellular Networks}}, year = 2022 }
• 
  Proactive Critical Event Prediction based on Monitoring Data with Focus on Technical Systems. M. O. Züfle; Thesis; Universität Würzburg. (2022).
  • [ BibTeX ]
  • [ Download ]
  @phdthesis{Zuefle2022, author = {Z{\"u}fle, Marwin Otto}, keywords = {descartes}, school = {Universit{\"a}t W{\"u}rzburg}, title = {Proactive Critical Event Prediction based on Monitoring Data with Focus on Technical Systems}, type = {doctoralthesis}, year = 2022 }
• 
  Self-Aware Optimization of Cyber-Physical Systems in Intelligent Transportation and Logistics Systems. V. Lesch; Thesis; Universität Würzburg. (2022, April).
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Le2022-Dissertation, author = {Lesch, Veronika}, keywords = {internet_of_things}, month = {04}, school = {Universität Würzburg}, title = {Self-Aware Optimization of Cyber-Physical Systems in Intelligent Transportation and Logistics Systems}, type = {Dissertation}, year = 2022 }
• 
  Identification of clinically distinct and prognostically important phenogroups in patients with acute heart failure by a machine learning approach. J. J. Albert; R. Leppich; V. Borst; F. Sahiti; N. Scholz; V. Cejka; C. Morbach; G. Ertl; C. Angermann; S. Frantz; others; in EUROPEAN JOURNAL OF HEART FAILURE (2022). (Vol. 24) 179–179.
  • [ BibTeX ]
  @inproceedings{albert2022identification, author = {Albert, J Judith and Leppich, R and Borst, V and Sahiti, F and Scholz, N and Cejka, V and Morbach, C and Ertl, G and Angermann, C and Frantz, S and others}, booktitle = {EUROPEAN JOURNAL OF HEART FAILURE}, keywords = {t_interdisciplinary}, organization = {WILEY 111 RIVER ST, HOBOKEN 07030-5774, NJ USA}, pages = {179–179}, title = {Identification of clinically distinct and prognostically important phenogroups in patients with acute heart failure by a machine learning approach}, volume = 24, year = 2022 }
• 
  Digital Contact Tracing Solutions: Promises, Pitfalls and Challenges. N. Thien Duc; M. Markus; D. Alexandra; S. Ahmad-Reza; V. Ivan; in ArXiv I arXiv 2202.06698v2 (2022).
  • [ BibTeX ]
  • [ URL ]
  @article{thienduc2022digital, author = {Thien Duc, Nguyen and Markus, Miettinen and Alexandra, Dmitrienko and Ahmad-Reza, Sadeghi and Ivan, Visconti}, journal = {ArXiv I arXiv 2202.06698v2}, keywords = {TraceCORONA}, month = 10, title = {Digital Contact Tracing Solutions: Promises, Pitfalls and Challenges}, year = 2022 }
• 
  Measurement, Modeling, and Emulation of Power Consumption of Distributed Systems. N. Schmitt; (2022, June).
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Schmitt_2022, author = {Schmitt, Norbert}, keywords = {power}, month = {06}, title = {Measurement, Modeling, and Emulation of Power Consumption of Distributed Systems}, year = 2022 }
• 
  Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning. A. Bauer; M. Straesser; L. Beierlieb; M. Meissner; S. Kounev; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
  • [ BibTeX ]
  • [ Download ]
  @conference{bauer2022automated, author = {Bauer, André and Straesser, Martin and Beierlieb, Lukas and Meissner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {machine_learning}, month = {04}, publisher = {ACM}, title = {Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning}, year = 2022 }
• 
  SPEC Research Group Newsletter, Issue 11. S. Kounev; A. van Hoorn; M. Straesser; A. Bauer; (2022, April).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-11, author = {Kounev, Samuel and van Hoorn, Andre and Straesser, Martin and Bauer, Andr{é}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {SPEC Research Group Newsletter, Issue 11}, year = 2022 }
• 
  SPEC Research - Introducing the Predictive Data Analytics Working Group. A. Bauer; M. Leznik; D. Seybold; I. Trubin; B. Erb; J. Domaschka; P. Jamshidi; in Companion of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
  • [ BibTeX ]
  • [ Download ]
  @conference{noauthororeditor, author = {Bauer, André and Leznik, Mark and Seybold, Daniel and Trubin, Igor and Erb, Benjamin and Domaschka, Jörg and Jamshidi, Pooyan}, booktitle = {Companion of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {spec}, month = {04}, title = {SPEC Research - Introducing the Predictive Data Analytics Working Group}, year = 2022 }
• 
  Change Point Detection for MongoDB Time Series Performance Regression. M. Leznik; M. S. Iqbal; I. A. Trubin; A. Lochner; P. Jamshidi; A. Bauer; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
  • [ BibTeX ]
  @conference{leznik2022change, author = {Leznik, Mark and Iqbal, Md Shahriar and Trubin, Igor A. and Lochner, Arne and Jamshidi, Pooyan and Bauer, André}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {performance}, month = {04}, publisher = {ACM}, title = {Change Point Detection for MongoDB Time Series Performance Regression}, year = 2022 }
• 
  SPEC Research - Introducing the Predictive Data Analytics Working Group. A. Bauer; M. Leznik; D. Seybold; I. Trubin; B. Erb; J. Domaschka; P. Jamshidi; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022).
  • [ BibTeX ]
  @conference{bauer2022research, author = {Bauer, André and Leznik, Mark and Seybold, Daniel and Trubin, Igor and Erb, Benjamin and Domaschka, Jörg and Jamshidi, Pooyan}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {spec}, month = {04}, series = {ICPE '22}, title = {SPEC Research - Introducing the Predictive Data Analytics Working Group}, year = 2022 }
• 
  FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles. M. Hadry; V. Lesch; S. Kounev; in Companion of the 13th ACM/SPEC International Conference on Performance Engineering (ICPE 2022) (2022).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{hadry2022towards, author = {Hadry, Marius and Lesch, Veronika and Kounev, Samuel}, booktitle = {Companion of the 13th ACM/SPEC International Conference on Performance Engineering (ICPE 2022)}, keywords = {cyber-physical_systems}, month = {04}, publisher = {ACM}, title = {FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles}, year = 2022 }
• 
  SPEC Efficiency Benchmark Development: How to Contribute to the Future of Energy Conservation. M. Meissner; K.-D. Lange; J. Arnold; S. Sharma; R. Tipley; N. Rawtani; D. Reiner; M. Petrich; A. Cragin; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  A driving force behind the improvement of server efficiency in recent years is the use of SPEC benchmarks. They are used in mandatory government regulations, the ISO/IEC 21836:2020 standard, and product marketing, giving server manufacturers and buyers significant incentive to improve energy efficiency. To produce relevant results, benchmarks need to take into account future trends in hardware and software development, such as the introduction of new accelerators and workloads. To keep pace with the development of the fast moving IT landscape, SPEC plans to introduce a workload bounty program to encourage researchers to develop novel workloads. Submitted workloads will be considered for inclusion in future SPEC Efficiency benchmarks and rewarded. In this paper, we outline the process of energy-efficiency benchmark development. SPEC ensures the development of high-quality benchmarks for government regulations through its extensive experience and collaboration with stakeholders from industry, academia, and governments. One of the tools that emerged from this process is the Chauffeur Worklet Development Kit (WDK), which can be used by researchers to develop next-generation workloads to enhance the real-world relevance of future SPEC benchmarks, a critical element for the benchmarks to contribute to future energy conservation.

  @inproceedings{meissner2022efficiency, abstract = {A driving force behind the improvement of server efficiency in recent years is the use of SPEC benchmarks. They are used in mandatory government regulations, the ISO/IEC 21836:2020 standard, and product marketing, giving server manufacturers and buyers significant incentive to improve energy efficiency. To produce relevant results, benchmarks need to take into account future trends in hardware and software development, such as the introduction of new accelerators and workloads. To keep pace with the development of the fast moving IT landscape, SPEC plans to introduce a workload bounty program to encourage researchers to develop novel workloads. Submitted workloads will be considered for inclusion in future SPEC Efficiency benchmarks and rewarded. In this paper, we outline the process of energy-efficiency benchmark development. SPEC ensures the development of high-quality benchmarks for government regulations through its extensive experience and collaboration with stakeholders from industry, academia, and governments. One of the tools that emerged from this process is the Chauffeur Worklet Development Kit (WDK), which can be used by researchers to develop next-generation workloads to enhance the real-world relevance of future SPEC benchmarks, a critical element for the benchmarks to contribute to future energy conservation.}, address = {New York, NY, USA}, author = {Meissner, Maximilian and Lange, Klaus-Dieter and Arnold, Jeremy and Sharma, Sanjay and Tipley, Roger and Rawtani, Nishant and Reiner, David and Petrich, Mike and Cragin, Aaron}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {spec}, month = {04}, publisher = {ACM}, title = {SPEC Efficiency Benchmark Development: How to Contribute to the Future of Energy Conservation}, year = 2022 }
• 
  Model Learning for Performance Prediction of Cloud-native Microservice Applications. J. Grohmann; Thesis; Universität Würzburg. (2022, March).

  Distinction
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  One consequence of the recent coronavirus pandemic is increased demand and use of online services around the globe. At the same time, performance requirements for modern technologies are becoming more stringent as users become accustomed to higher standards. These increased performance and availability requirements, coupled with the unpredictable usage growth, are driving an increasing proportion of applications to run on public cloud platforms as they promise better scalability and reliability. With data centers already responsible for about one percent of the world's power consumption, optimizing resource usage is of paramount importance. Simultaneously, meeting the increasing and changing resource and performance requirements is only possible by optimizing resource management without introducing additional overhead. This requires the research and development of new modeling approaches to understand the behavior of running applications with minimal information. However, the emergence of modern software paradigms makes it increasingly difficult to derive such models and renders previous performance modeling techniques infeasible. Modern cloud applications are often deployed as a collection of fine-grained and interconnected components called microservices. Microservice architectures offer massive benefits but also have broad implications for the performance characteristics of the respective systems. In addition, the microservices paradigm is typically paired with a DevOps culture, resulting in frequent application and deployment changes. Such applications are often referred to as cloud-native applications. In summary, the increasing use of ever-changing cloud-hosted microservice applications introduces a number of unique challenges for modeling the performance of modern applications. These include the amount, type, and structure of monitoring data, frequent behavioral changes, or infrastructure variabilities. This violates common assumptions of the state of the art and opens a research gap for our work. In this thesis, we present five techniques for automated learning of performance models for cloud-native software systems. We achieve this by combining machine learning with traditional performance modeling techniques. Unlike previous work, our focus is on cloud-hosted and continuously evolving microservice architectures, so-called cloud-native applications. Therefore, our contributions aim to solve the above challenges to deliver automated performance models with minimal computational overhead and no manual intervention. Depending on the cloud computing model, privacy agreements, or monitoring capabilities of each platform, we identify different scenarios where performance modeling, prediction, and optimization techniques can provide great benefits. Specifically, the contributions of this thesis are as follows: Monitorless: Application-agnostic prediction of performance degradations. To manage application performance with only platform-level monitoring, we propose Monitorless, the first truly application-independent approach to detecting performance degradation. We use machine learning to bridge the gap between platform-level monitoring and application-specific measurements, eliminating the need for application-level monitoring. Monitorless creates a single and holistic resource saturation model that can be used for heterogeneous and untrained applications. Results show that Monitorless infers resource-based performance degradation with 97% accuracy. Moreover, it can achieve similar performance to typical autoscaling solutions, despite using less monitoring information. SuanMing: Predicting performance degradation using tracing. We introduce SuanMing to mitigate performance issues before they impact the user experience. This contribution is applied in scenarios where tracing tools enable application-level monitoring. SuanMing predicts explainable causes of expected performance degradations and prevents performance degradations before they occur. Evaluation results show that SuanMing can predict and pinpoint future performance degradations with an accuracy of over 90%. SARDE: Continuous and autonomous estimation of resource demands. We present SARDE to learn application models for highly variable application deployments. This contribution focuses on the continuous estimation of application resource demands, a key parameter of performance models. SARDE represents an autonomous ensemble estimation technique. It dynamically and continuously optimizes, selects, and executes an ensemble of approaches to estimate resource demands in response to changes in the application or its environment. Through continuous online adaptation, SARDE efficiently achieves an average resource demand estimation error of 15.96% in our evaluation. DepIC: Learning parametric dependencies from monitoring data. DepIC utilizes feature selection techniques in combination with an ensemble regression approach to automatically identify and characterize parametric dependencies. Although parametric dependencies can massively improve the accuracy of performance models, DepIC is the first approach to automatically learn such parametric dependencies from passive monitoring data streams. Our evaluation shows that DepIC achieves 91.7% precision in identifying dependencies and reduces the characterization prediction error by 30% compared to the best individual approach. Baloo: Modeling the configuration space of databases. To study the impact of different configurations within distributed DBMSs, we introduce Baloo. Our last contribution models the configuration space of databases considering measurement variabilities in the cloud. More specifically, Baloo dynamically estimates the required benchmarking measurements and automatically builds a configuration space model of a given DBMS. Our evaluation of Baloo on a dataset consisting of 900 configuration points shows that the framework achieves a prediction error of less than 11% while saving up to 80% of the measurement effort. Although the contributions themselves are orthogonally aligned, taken together they provide a holistic approach to performance management of modern cloud-native microservice applications. Our contributions are a significant step forward as they specifically target novel and cloud-native software development and operation paradigms, surpassing the capabilities and limitations of previous approaches. In addition, the research presented in this paper also has a significant impact on the industry, as the contributions were developed in collaboration with research teams from Nokia Bell Labs, Huawei, and Google. Overall, our solutions open up new possibilities for managing and optimizing cloud applications and improve cost and energy efficiency.

  @phdthesis{Gr2022-Dissertation, abstract = {One consequence of the recent coronavirus pandemic is increased demand and use of online services around the globe. At the same time, performance requirements for modern technologies are becoming more stringent as users become accustomed to higher standards. These increased performance and availability requirements, coupled with the unpredictable usage growth, are driving an increasing proportion of applications to run on public cloud platforms as they promise better scalability and reliability. With data centers already responsible for about one percent of the world's power consumption, optimizing resource usage is of paramount importance. Simultaneously, meeting the increasing and changing resource and performance requirements is only possible by optimizing resource management without introducing additional overhead. This requires the research and development of new modeling approaches to understand the behavior of running applications with minimal information. However, the emergence of modern software paradigms makes it increasingly difficult to derive such models and renders previous performance modeling techniques infeasible. Modern cloud applications are often deployed as a collection of fine-grained and interconnected components called microservices. Microservice architectures offer massive benefits but also have broad implications for the performance characteristics of the respective systems. In addition, the microservices paradigm is typically paired with a DevOps culture, resulting in frequent application and deployment changes. Such applications are often referred to as cloud-native applications. In summary, the increasing use of ever-changing cloud-hosted microservice applications introduces a number of unique challenges for modeling the performance of modern applications. These include the amount, type, and structure of monitoring data, frequent behavioral changes, or infrastructure variabilities. This violates common assumptions of the state of the art and opens a research gap for our work. In this thesis, we present five techniques for automated learning of performance models for cloud-native software systems. We achieve this by combining machine learning with traditional performance modeling techniques. Unlike previous work, our focus is on cloud-hosted and continuously evolving microservice architectures, so-called cloud-native applications. Therefore, our contributions aim to solve the above challenges to deliver automated performance models with minimal computational overhead and no manual intervention. Depending on the cloud computing model, privacy agreements, or monitoring capabilities of each platform, we identify different scenarios where performance modeling, prediction, and optimization techniques can provide great benefits. Specifically, the contributions of this thesis are as follows: Monitorless: Application-agnostic prediction of performance degradations. To manage application performance with only platform-level monitoring, we propose Monitorless, the first truly application-independent approach to detecting performance degradation. We use machine learning to bridge the gap between platform-level monitoring and application-specific measurements, eliminating the need for application-level monitoring. Monitorless creates a single and holistic resource saturation model that can be used for heterogeneous and untrained applications. Results show that Monitorless infers resource-based performance degradation with 97% accuracy. Moreover, it can achieve similar performance to typical autoscaling solutions, despite using less monitoring information. SuanMing: Predicting performance degradation using tracing. We introduce SuanMing to mitigate performance issues before they impact the user experience. This contribution is applied in scenarios where tracing tools enable application-level monitoring. SuanMing predicts explainable causes of expected performance degradations and prevents performance degradations before they occur. Evaluation results show that SuanMing can predict and pinpoint future performance degradations with an accuracy of over 90%. SARDE: Continuous and autonomous estimation of resource demands. We present SARDE to learn application models for highly variable application deployments. This contribution focuses on the continuous estimation of application resource demands, a key parameter of performance models. SARDE represents an autonomous ensemble estimation technique. It dynamically and continuously optimizes, selects, and executes an ensemble of approaches to estimate resource demands in response to changes in the application or its environment. Through continuous online adaptation, SARDE efficiently achieves an average resource demand estimation error of 15.96% in our evaluation. DepIC: Learning parametric dependencies from monitoring data. DepIC utilizes feature selection techniques in combination with an ensemble regression approach to automatically identify and characterize parametric dependencies. Although parametric dependencies can massively improve the accuracy of performance models, DepIC is the first approach to automatically learn such parametric dependencies from passive monitoring data streams. Our evaluation shows that DepIC achieves 91.7% precision in identifying dependencies and reduces the characterization prediction error by 30% compared to the best individual approach. Baloo: Modeling the configuration space of databases. To study the impact of different configurations within distributed DBMSs, we introduce Baloo. Our last contribution models the configuration space of databases considering measurement variabilities in the cloud. More specifically, Baloo dynamically estimates the required benchmarking measurements and automatically builds a configuration space model of a given DBMS. Our evaluation of Baloo on a dataset consisting of 900 configuration points shows that the framework achieves a prediction error of less than 11% while saving up to 80% of the measurement effort. Although the contributions themselves are orthogonally aligned, taken together they provide a holistic approach to performance management of modern cloud-native microservice applications. Our contributions are a significant step forward as they specifically target novel and cloud-native software development and operation paradigms, surpassing the capabilities and limitations of previous approaches. In addition, the research presented in this paper also has a significant impact on the industry, as the contributions were developed in collaboration with research teams from Nokia Bell Labs, Huawei, and Google. Overall, our solutions open up new possibilities for managing and optimizing cloud applications and improve cost and energy efficiency.}, author = {Grohmann, Johannes}, keywords = {performance}, month = {03}, note = {Distinction}, school = {Universität Würzburg}, title = {Model Learning for Performance Prediction of Cloud-native Microservice Applications}, type = {Dissertation}, year = 2022 }
• 
  FedCRI: Federated Mobile Cyber-Risk Intelligence. H. Fereidooni; A. Dmitrienko; F. Madlener; P. Rieger; M. Miettinen; A.-R. Sadeghi; in The Network and Distributed System Security Symposium (NDSS) (2022).

  (to appear)
  • [ BibTeX ]
  @inproceedings{fereidooni2022fedcri, author = {Fereidooni, Hossein and Dmitrienko, Alexandra and Madlener, Felix and Rieger, Phillip and Miettinen, Markus and Sadeghi, Ahmad-Reza}, booktitle = {The Network and Distributed System Security Symposium (NDSS)}, keywords = {security}, note = {(to appear)}, title = {FedCRI: Federated Mobile Cyber-Risk Intelligence}, year = 2022 }
• 
  An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case. T. Prantl; S. Engel; A. Bauer; A. E. Ben Yahya; S. Herrnleben; L. Iffländer; A. Dmitrienko; S. Kounev; in 2022 IEEE 95th Vehicular Technology Conference (VTC2022) (2022).
  • [ BibTeX ]
  @inproceedings{PrEnBaAlHeIfDmKo2022-IoT/CPS-DistributedGroupEncryption, author = {Prantl, Thomas and Engel, Simon and Bauer, Andre and Ben Yahya, Ala Eddine and Herrnleben, Stefan and Iffländer, Lukas and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {2022 IEEE 95th Vehicular Technology Conference (VTC2022)}, keywords = {t_workshop}, organization = {IEEE}, title = {An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case}, year = 2022 }
• 
  A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions. E. Henrichs; V. Lesch; M. Straesser; S. Kounev; C. Krupitzer; in Information and Software Technology (2022). 149 106940.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.

  @article{HENRICHS2022106940, abstract = {Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.}, author = {Henrichs, Elia and Lesch, Veronika and Straesser, Martin and Kounev, Samuel and Krupitzer, Christian}, journal = {Information and Software Technology}, keywords = {sys:relevantfor:se-group}, pages = 106940, title = {A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions}, volume = 149, year = 2022 }
• 
  Towards Forecasting Future Snow Cover Dynamics in the European Alps - The Potential of Long Optical Remote-Sensing Time Series. J. Koehler; A. Bauer; A. J. Dietz; C. Kuenzer; in Remote Sensing (2022). 14(18)
  • [ BibTeX ]
  • [ URL ]
  @article{rs14184461, author = {Koehler, Jonas and Bauer, André and Dietz, Andreas J. and Kuenzer, Claudia}, journal = {Remote Sensing}, keywords = {t_journalmagazine}, number = 18, title = {Towards Forecasting Future Snow Cover Dynamics in the European Alps - The Potential of Long Optical Remote-Sensing Time Series}, volume = 14, year = 2022 }
• 
  A case study on the stability of performance tests for serverless applications. S. Eismann; D. Costa; L. Liao; C.-P. Bezemer; W. Shang; A. van Hoorn; S. Kounev; in Journal of Systems and Software (JSS) (2022). 189
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.

  @article{eismann2022study, abstract = {Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.}, author = {Eismann, Simon and Costa, Diego and Liao, Lizhi and Bezemer, Cor-Paul and Shang, Weiyi and van Hoorn, André and Kounev, Samuel}, journal = {Journal of Systems and Software (JSS)}, keywords = {t_journalmagazine}, title = {A case study on the stability of performance tests for serverless applications}, volume = 189, year = 2022 }
• 
  Edge Workload Trace Gathering and Analysis for Benchmarking. K. Toczé; N. Schmitt; U. Kargén; A. Aral; I. Brandi’c}}; in 2022 IEEE 6th International Conference on Fog and Edge Computing (ICFEC) (2022). 34–41.
  • [ BibTeX ]
  @inproceedings{tocze2022edge, author = {Tocz{\'e}, Klervie and Schmitt, Norbert and Karg{\'e}n, Ulf and Aral, Atakan and Brandi{\'c}, Ivona}, booktitle = {2022 IEEE 6th International Conference on Fog and Edge Computing (ICFEC)}, keywords = {descartes}, organization = {IEEE}, pages = {34–41}, title = {Edge Workload Trace Gathering and Analysis for Benchmarking}, year = 2022 }
• 
  Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency. M. Meissner; S. Kamthania; N. Rawtani; J. Bucek; K.-D. Lange; S. Kounev; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022).
  • [ BibTeX ]
  @inproceedings{meissner2022experience, address = {New York, NY, USA}, author = {Meissner, Maximilian and Kamthania, Supriya and Rawtani, Nishant and Bucek, James and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {t_workshop}, month = {04}, publisher = {ACM}, title = {Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency}, year = 2022 }
• 
  The Relationship Between the Distribution of Training Intensity and Performance of Kayak and Canoe Sprinters: A Retrospective Observational Analysis of One Season of Competition. M. Matzka; R. Leppich; H.-C. Holmberg; B. Sperlich; C. Zinner; in Frontiers in Sports and Active Living (2022). 3 788108.
  • [ BibTeX ]
  @article{matzka2022relationship, author = {Matzka, Manuel and Leppich, Robert and Holmberg, Hans-Christer and Sperlich, Billy and Zinner, Christoph}, journal = {Frontiers in Sports and Active Living}, keywords = {t_journalmagazine}, pages = 788108, publisher = {Frontiers Media SA}, title = {The Relationship Between the Distribution of Training Intensity and Performance of Kayak and Canoe Sprinters: A Retrospective Observational Analysis of One Season of Competition}, volume = 3, year = 2022 }

• 
  A New Course on Systems Benchmarking-For Scientists and Engineers S. Kounev; (2021). 127–127.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @presentation{kounev2021new, author = {Kounev, Samuel}, booktitle = {Companion of the ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {127–127}, title = {A New Course on Systems Benchmarking-For Scientists and Engineers}, year = 2021 }
• 
  Increasing Security in Satellite Networks. K. Schilling; A. Dmitrienko; in 72nd International Astronautical Congress (IAC) (2021).
  • [ BibTeX ]
  @inproceedings{schilling2021increasing, author = {Schilling, Klaus and Dmitrienko, Alexandra}, booktitle = {72nd International Astronautical Congress (IAC)}, keywords = {security}, title = {Increasing Security in Satellite Networks}, year = 2021 }
• 
  An Overview on Approaches for Coordination of Platoons. V. Lesch; M. Breitbach; M. Segata; C. Becker; S. Kounev; C. Krupitzer; in IEEE Transactions on Intelligent Transportation Systems (2021).

  Early Access on IEEE Xplore
  • [ BibTeX ]
  • [ Download ]
  @article{lesch2021overview, author = {Lesch, Veronika and Breitbach, Martin and Segata, Michele and Becker, Christian and Kounev, Samuel and Krupitzer, Christian}, journal = {IEEE Transactions on Intelligent Transportation Systems}, keywords = {platooning}, note = {Early Access on IEEE Xplore}, title = {An Overview on Approaches for Coordination of Platoons}, year = 2021 }
• 
  RIP StrandHogg: A Practical StrandHogg Attack Detection Method on Android. J. Stang; A. Dmitrienko; S. Roth; in 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec) (2021).
  • [ Abstract ]
  • [ BibTeX ]
  StrandHogg vulnerabilities affect Android’s multitasking system and threaten up to 90% of Android platforms, which translates to millions of affected users. Existing countermeasures require modification of the OS, have usability drawbacks, or are limited to the detection of certain attack versions. In this work, we aim to develop a generic, efficient, and usability-friendly attack detection method, which does not require OS modifications and can be employed by apps installed on any vulnerable Android platform. To achieve our goal, we analyze StrandHogg attack techniques and develop two countermeasures, one using Machine Learning and the other one using ActivityCounter – a reliable attack indicator, which we could synthetically engineer. Our first approach achieves an average F1 score of 92% across all attack variations, while ActivityCounter shows superior performance and efficiently detects all attack versions without false positives. ActivityCounter is the first solution without practical limitations, which can be easily deployed in practice and protect millions of affected users.

  @inproceedings{stang2021strandhogg, abstract = {StrandHogg vulnerabilities affect Android’s multitasking system and threaten up to 90% of Android platforms, which translates to millions of affected users. Existing countermeasures require modification of the OS, have usability drawbacks, or are limited to the detection of certain attack versions. In this work, we aim to develop a generic, efficient, and usability-friendly attack detection method, which does not require OS modifications and can be employed by apps installed on any vulnerable Android platform. To achieve our goal, we analyze StrandHogg attack techniques and develop two countermeasures, one using Machine Learning and the other one using ActivityCounter – a reliable attack indicator, which we could synthetically engineer. Our first approach achieves an average F1 score of 92% across all attack variations, while ActivityCounter shows superior performance and efficiently detects all attack versions without false positives. ActivityCounter is the first solution without practical limitations, which can be easily deployed in practice and protect millions of affected users.}, author = {Stang, Jasper and Dmitrienko, Alexandra and Roth, Sascha}, booktitle = {14th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec)}, keywords = {Android}, title = {RIP StrandHogg: A Practical StrandHogg Attack Detection Method on Android}, year = 2021 }
• 
  Teaching Software Testing Using Automated Grading. L. Beierlieb; L. Iffländer; T. Schneider; T. Prantl; S. Kounev; in Proceedings of the Fifth Workshop "Automatische Bewertung von Programmieraufgaben" (ABP 2021),virtual event, October 28-29, 2021, A. Greubel, S. Strickroth, M. Striewe (eds.) (2021).
  • [ BibTeX ]
  @inproceedings{Beierlieb_2021_TeachingSoftwareTestingUsingAutomatedGrading, author = {Beierlieb, Lukas and Iffländer, Lukas and Schneider, Tobias and Prantl, Thomas and Kounev, Samuel}, booktitle = {Proceedings of the Fifth Workshop "Automatische Bewertung von Programmieraufgaben" (ABP 2021),virtual event, October 28-29, 2021}, editor = {Greubel, André and Strickroth, Sven and Striewe, Michael}, keywords = {sys:relevantfor:se-group}, publisher = {Gesellschaft für Informatik}, title = {Teaching Software Testing Using Automated Grading}, year = 2021 }
• 
  Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies. V. Lesch; T. Noack; J. Hefter; S. Kounev; C. Krupitzer; in Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021 (2021).

  Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{lesch2021towards, author = {Lesch, Veronika and Noack, Tanja and Hefter, Johannes and Kounev, Samuel and Krupitzer, Christian}, booktitle = {Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021}, keywords = {internet_of_things}, note = {Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%}, publisher = {IEEE}, title = {Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies}, year = 2021 }
• 
  Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms. V. Lesch; M. König; S. Kounev; A. Stein; C. Krupitzer; in Applied Intelligence (2021). 52 9476–9500.
  • [ BibTeX ]
  • [ DOI ]
  @article{lesch2021tackling, author = {Lesch, Veronika and König, Maximilian and Kounev, Samuel and Stein, Anthony and Krupitzer, Christian}, journal = {Applied Intelligence}, keywords = {t_journalmagazine}, pages = {9476–9500}, title = {Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms}, volume = 52, year = 2021 }
• 
  A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools. M. Züfle; F. Moog; V. Lesch; C. Krupitzer; S. Kounev; in ISA Transactions: The Journal of Automation (2021).

  (in press)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.

  @article{zuefle2021anomalydetection, abstract = {Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.}, author = {Züfle, Marwin and Moog, Felix and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, journal = {ISA Transactions: The Journal of Automation}, keywords = {t_journalmagazine}, note = {(in press)}, title = {A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools}, year = 2021 }
• 
  Container Start Times: Empirical Analysis and Predictability M. Straesser; (2021).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @presentation{straesser2021container, author = {Straesser, Martin}, keywords = {empirical_study}, title = {Container Start Times: Empirical Analysis and Predictability}, year = 2021 }
• 
  Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping. N. Finke; M. Mohr; A. Lontke; M. Züfle; S. Kounev; R. Möller; in Research Challenges in Information Science (RCIS) (2021). 189–204.
  • [ BibTeX ]
  @inproceedings{finke2021recommendations, author = {Finke, Nils and Mohr, Marisa and Lontke, Alexander and Z{\"u}fle, Marwin and Kounev, Samuel and M{\"o}ller, Ralf}, booktitle = {Research Challenges in Information Science (RCIS)}, keywords = {t_interdisciplinary}, organization = {Springer}, pages = {189–204}, title = {Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping}, year = 2021 }
• 
  Establishing a cardiac training group for patients with heart failure: the “HIP-in-Würzburg” study. G. Güder; J. Wilkesmann; N. Scholz; R. Leppich; P. Düking; B. Sperlich; C. Rost; S. Frantz; C. Morbach; F. Sahiti; others; in Clinical Research in Cardiology (2021). 1–10.
  • [ BibTeX ]
  @article{guder2021establishing, author = {G{\"u}der, G{\"u}lmisal and Wilkesmann, Joana and Scholz, Nina and Leppich, Robert and D{\"u}king, Peter and Sperlich, Billy and Rost, Christian and Frantz, Stefan and Morbach, Caroline and Sahiti, Floran and others}, journal = {Clinical Research in Cardiology}, keywords = {t_journalmagazine}, pages = {1–10}, publisher = {Springer}, title = {Establishing a cardiac training group for patients with heart failure: the “HIP-in-W{\"u}rzburg” study}, year = 2021 }
• 
  ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations. S. Herrnleben; M. Leidinger; V. Lesch; T. Prantl; J. Grohmann; C. Krupitzer; S. Kounev; in Performance Evaluation Methodologies and Tools, Q. Zhao, L. Xia (eds.) (2021). 72–92.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.

  @inproceedings{10.1007/978-3-030-92511-6_5, abstract = {Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Leidinger, Maximilian and Lesch, Veronika and Prantl, Thomas and Grohmann, Johannes and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Performance Evaluation Methodologies and Tools}, editor = {Zhao, Qianchuan and Xia, Li}, keywords = {networking}, pages = {72–92}, publisher = {Springer International Publishing}, title = {ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations}, year = 2021 }
• 
  Software Testing Strategies for Detecting Hypercall Handlerstextquotesingle Aging-related Bugs. L. Beierlieb; A. Avritzer; L. Ifflander; N. Antunes; A. Milenkoski; S. Kounev; in 2021 IEEE International Symposium on Software Reliability Engineering Workshops ({ISSREW}) (2021).
  • [ BibTeX ]
  @inproceedings{Beierlieb_2021_SoftwareTestingStrategiesforDetectingHypercallHandlersAgingRelatedBugs, author = {Beierlieb, Lukas and Avritzer, Alberto and Ifflander, Lukas and Antunes, Nuno and Milenkoski, Aleksandar and Kounev, Samuel}, booktitle = {2021 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})}, keywords = {security}, publisher = {{IEEE}}, title = {Software Testing Strategies for Detecting Hypercall Handlers{\textquotesingle} Aging-related Bugs}, year = 2021 }
• 
  Automatisierte Hybride Zeitreihenprognose. A. Bauer; in Ausgezeichnete Informatikdissertationen 2020, S. Hölldobler (ed.) (2021). 29–38.
  • [ BibTeX ]
  @inproceedings{mci/Bauer2021, address = {Bonn}, author = {Bauer, André}, booktitle = {Ausgezeichnete Informatikdissertationen 2020}, editor = {Hölldobler, Steffen}, keywords = {telescope}, pages = {29-38}, publisher = {Gesellschaft für Informatik e.V.}, title = {Automatisierte Hybride Zeitreihenprognose}, year = 2021 }
• 
  Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT. T. Prantl; D. Prantl; A. Bauer; L. Iffländer; A. Dmitrenko; C. Krupitzer; S. Kounev; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
  • [ BibTeX ]
  @inproceedings{PrPrBaIfDmKrKo2021-IPCCC-QuantumGroupEncryption, author = {Prantl, Thomas and Prantl, Dominik and Bauer, André and Iffländer, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {sss-group}, title = {Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT}, year = 2021 }
• 
  The integration of training and off-training activities substantially alters training volume and load analysis in elite rowers. G. Treff; R. Leppich; K. Winkert; J. M. Steinacker; B. Mayer; B. Sperlich; in Scientific reports (2021). 11(1) 1–10.
  • [ BibTeX ]
  @article{treff2021integration, author = {Treff, Gunnar and Leppich, Robert and Winkert, Kay and Steinacker, J{\"u}rgen M and Mayer, Benjamin and Sperlich, Billy}, journal = {Scientific reports}, keywords = {t_journalmagazine}, number = 1, pages = {1–10}, publisher = {Nature Publishing Group}, title = {The integration of training and off-training activities substantially alters training volume and load analysis in elite rowers}, volume = 11, year = 2021 }
• 
  Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction. M. Züfle; F. Erhard; S. Kounev; in 20th IEEE International Conference on Machine Learning and Applications (ICMLA) (2021).

  (to appear)
  • [ BibTeX ]
  @inproceedings{zufle2021machine, author = {Z{\"u}fle, Marwin and Erhard, Florian and Kounev, Samuel}, booktitle = {20th IEEE International Conference on Machine Learning and Applications (ICMLA)}, keywords = {online_monitoring_and_forecasting}, note = {(to appear)}, title = {Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction}, year = 2021 }
• 
  Sound Smart Contract Security Testing with Just One Tool. A. Dmitrienko; H. Chen; H. Fereidooni; C. Sendner; A.-R. Sadeghi; F. Koushanfar; in CyberSec&AI 2021 (2021).
  • [ BibTeX ]
  • [ URL ]
  @conference{dmitrienko2021sound, author = {Dmitrienko, Alexandra and Chen, Huili and Fereidooni, Hossein and Sendner, Christoph and Sadeghi, Ahmad-Reza and Koushanfar, Farinaz}, booktitle = {CyberSec&AI 2021}, keywords = {sss-group}, title = {Sound Smart Contract Security Testing with Just One Tool}, year = 2021 }
• 
  A Case Study of Vehicle Route Optimization V. Lesch; M. König; S. Kounev; A. Stein; C. Krupitzer; (2021).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @techreport{lesch2021studyvrp, author = {Lesch, Veronika and König, Maximilian and Kounev, Samuel and Stein, Anthony and Krupitzer, Christian}, institution = {Universit{ä}t W{ü}rzburg and PASS Logistics Solutions AG and Universit{ä}t Hohenheim}, keywords = {t_techreport}, title = {A Case Study of Vehicle Route Optimization}, year = 2021 }
• 
  Serverless Computing (Dagstuhl Seminar 21201). C. Abad; I. T. Foster; N. Herbst; A. Iosup; in Dagstuhl Reports, (C. Abad; I. T. Foster; N. Herbst; A. Iosup, eds.) (2021). 11(4) 34–93.
  • [ BibTeX ]
  • [ URL ]
  @article{abad_et_al:DagRep.11.4.34, address = {Dagstuhl, Germany}, annote = {Keywords: Cloud computing, Cloud continuum, data-driven, design patterns, DevOps, experimentation, model-driven, serverless computing, simulation, software architecture, systems management, vision}, author = {Abad, Cristina and Foster, Ian T. and Herbst, Nikolas and Iosup, Alexandru}, editor = {Abad, Cristina and Foster, Ian T. and Herbst, Nikolas and Iosup, Alexandru}, journal = {Dagstuhl Reports}, keywords = {resource_management}, number = 4, pages = {34–93}, publisher = {Schloss Dagstuhl – Leibniz-Zentrum f{\"u}r Informatik}, title = {{Serverless Computing (Dagstuhl Seminar 21201)}}, volume = 11, year = 2021 }
• 
  Auswirkungen der COVID-19-Pandemie auf Sportlerinnen und Sportler in Deutschland. C. Zinner; M. Matzka; R. Leppich; S. Kounev; H.-C. Holmberg; B. Sperlich; in 53. asp Jahrestagung (2021).
  • [ BibTeX ]
  @article{zinner2021auswirkungen, author = {Zinner, Christoph and Matzka, Manuel and Leppich, Robert and Kounev, Samuel and Holmberg, Hans-Christian and Sperlich, Billy}, journal = {53. asp Jahrestagung}, keywords = {t_interdisciplinary}, title = {Auswirkungen der COVID-19-Pandemie auf Sportlerinnen und Sportler in Deutschland}, year = 2021 }
• 
  A Case Study on Optimization of Warehouses. V. Lesch; P. Müller; M. Krämer; S. Kounev; C. Krupitzer; in arXiv preprint arXiv:2112.12058 (2021).
  • [ BibTeX ]
  @article{lesch2021case, author = {Lesch, Veronika and M{\"u}ller, Patrick and Kr{\"a}mer, Moritz and Kounev, Samuel and Krupitzer, Christian}, journal = {arXiv preprint arXiv:2112.12058}, keywords = {t_journalmagazine}, title = {A Case Study on Optimization of Warehouses}, year = 2021 }
• 
  Systems Benchmarking S. Kounev; (2021).
  • [ BibTeX ]
  @presentation{kounev2021systems, author = {Kounev, Samuel}, keywords = {descartes}, title = {Systems Benchmarking}, year = 2021 }
• 
  Automated Hybrid Time Series Forecasting: Design, Benchmarking, and Use Cases. A. Bauer; Thesis; Universität Würzburg. (2021).
  • [ BibTeX ]
  @phdthesis{Bauer2021-phdthesis, author = {Bauer, Andr{\'e}}, keywords = {descartes}, school = {Universit{\"a}t W{\"u}rzburg}, title = {Automated Hybrid Time Series Forecasting: Design, Benchmarking, and Use Cases}, type = {doctoralthesis}, year = 2021 }
• 
  The Science of Systems Benchmarking S. Kounev; (2021).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @presentation{kounev2021science, author = {Kounev, Samuel}, keywords = {descartes}, title = {The Science of Systems Benchmarking}, year = 2021 }
• 
  A Case Study on Optimization of Platooning Coordination V. Lesch; M. Hadry; S. Kounev; C. Krupitzer; (2021).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @techreport{lesch2021studyplatooning, author = {Lesch, Veronika and Hadry, Marius and Kounev, Samuel and Krupitzer, Christian}, institution = {Universit{ä}t W{ü}rzburg and Universit{ä}t Hohenheim}, keywords = {t_techreport}, title = {A Case Study on Optimization of Platooning Coordination}, year = 2021 }
• 
  A Case Study on Optimization of Warehouses V. Lesch; P. B. Müller; M. Krämer; S. Kounev; C. Krupitzer; (2021).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @techreport{lesch2021study, author = {Lesch, Veronika and Müller, Patrick B.M. and Krämer, Moritz and Kounev, Samuel and Krupitzer, Christian}, keywords = {t_techreport}, title = {A Case Study on Optimization of Warehouses}, year = 2021 }
• 
  All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers. C. Hagen; C. Weinert; C. Sendner; A. Dmitrienko; T. Schneider; in Network and Distributed System Security Symposium (NDSS) (2021).

  (To Appear)
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{DBLP:conf/ndss/2021, author = {Hagen, Christoph and Weinert, Christian and Sendner, Christoph and Dmitrienko, Alexandra and Schneider, Thomas}, booktitle = {Network and Distributed System Security Symposium (NDSS)}, keywords = {dblp}, note = {(To Appear)}, publisher = {The Internet Society}, title = {All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers}, year = 2021 }
• 
  SPEC - Spotlight on the International Standards Group ({ISG}). N. Schmitt; K.-D. Lange; S. Sharma; A. Cragin; D. Reiner; S. Kounev; in Companion of the ACM/SPEC International Conference on Performance Engineering (2021).
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{Schmitt_2021, author = {Schmitt, Norbert and Lange, Klaus-Dieter and Sharma, Sanjay and Cragin, Aaron and Reiner, David and Kounev, Samuel}, booktitle = {Companion of the {ACM}/{SPEC} International Conference on Performance Engineering}, keywords = {descartes}, month = {04}, publisher = {{ACM}}, title = {{SPEC} - Spotlight on the International Standards Group ({ISG})}, year = 2021 }
• 
  Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads. J. Domaschka; S. Eismann; M. Leznik; J. Grohmann; S. Kounev; D. Seybold; in Companion of the ACM/SPEC International Conference on Performance Engineering (2021). (Vol. ICPE ’21) 175–178.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Distributed Database Management Systems (DBMS) are a crucial component of modern IT applications. Understanding their performance and non-functional properties is of paramount importance. Yet, benchmarking distributed DBMS has proven to be difficult in practice. Either, a realistic workload is often mapped to a synthetic workload without knowing if this mapping is correct or available workload traces are replayed. While the latter approach provides more realistic results, real-world traces are hard to obtain and their scope is limited in time scale and variance.We propose collecting real-world traces and then applying data generation techniques to synthesize similar realistic traces based on it. Based in this approach, we can obtain workloads for benchmarking, exhibit variability with respect to different aspects of interest while still being similar to the original traces. Varying generation parameters, we are able to support benchmarking what-if scenarios with hypothetical workloads and introduced anomalies.

  @inproceedings{domaschka2021buzzy, abstract = {Distributed Database Management Systems (DBMS) are a crucial component of modern IT applications. Understanding their performance and non-functional properties is of paramount importance. Yet, benchmarking distributed DBMS has proven to be difficult in practice. Either, a realistic workload is often mapped to a synthetic workload without knowing if this mapping is correct or available workload traces are replayed. While the latter approach provides more realistic results, real-world traces are hard to obtain and their scope is limited in time scale and variance.We propose collecting real-world traces and then applying data generation techniques to synthesize similar realistic traces based on it. Based in this approach, we can obtain workloads for benchmarking, exhibit variability with respect to different aspects of interest while still being similar to the original traces. Varying generation parameters, we are able to support benchmarking what-if scenarios with hypothetical workloads and introduced anomalies.}, address = {New York, NY, USA}, author = {Domaschka, Jörg and Eismann, Simon and Leznik, Mark and Grohmann, Johannes and Kounev, Samuel and Seybold, Daniel}, booktitle = {Companion of the ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {175–178}, publisher = {Association for Computing Machinery}, title = {Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads}, volume = {ICPE '21}, year = 2021 }
• 
  Towards a Mobile Robot Localization Benchmark with Challenging Sensordata in an Industrial Environment. F. Spiess; J. Friesslich; D. Bluemm; F. Mast; D. Vinokour; S. Kounev; T. Kaupp; N. Strobel; in 2021 20th International Conference on Advanced Robotics (ICAR) (2021). 857–864.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  To arrive at a realistic assessment of localization methods in terms of their performance in an industrial environment under various challenging conditions, we provide a benchmark to evaluate algorithms both for individual components as well as multi-sensor systems. For several sensor types, including wheel odometry, RGB cameras, RGB-D cameras, and LIDAR, potential issues were identified. The accuracy of wheel odometry, for example, when there are bumps on the track. For each sensor type, we explicitly chose a track for the benchmark dataset containing situations where the sensor fails to provide adequate measurements. Based on the acquired sensor data, localization can be achieved either using a single sensor information or sensor fusion. To help evaluate the output of associated localization algorithms, we provide a software to evaluate a set of metrics as part of the paper. An example application of the benchmark with state-of-the-art algorithms for each sensor is also provided.

  @inproceedings{9659355, abstract = {To arrive at a realistic assessment of localization methods in terms of their performance in an industrial environment under various challenging conditions, we provide a benchmark to evaluate algorithms both for individual components as well as multi-sensor systems. For several sensor types, including wheel odometry, RGB cameras, RGB-D cameras, and LIDAR, potential issues were identified. The accuracy of wheel odometry, for example, when there are bumps on the track. For each sensor type, we explicitly chose a track for the benchmark dataset containing situations where the sensor fails to provide adequate measurements. Based on the acquired sensor data, localization can be achieved either using a single sensor information or sensor fusion. To help evaluate the output of associated localization algorithms, we provide a software to evaluate a set of metrics as part of the paper. An example application of the benchmark with state-of-the-art algorithms for each sensor is also provided.}, author = {Spiess, Florian and Friesslich, Jonas and Bluemm, Daniel and Mast, Fabio and Vinokour, Dmitrij and Kounev, Samuel and Kaupp, Tobias and Strobel, Norbert}, booktitle = {2021 20th International Conference on Advanced Robotics (ICAR)}, keywords = {t_interdisciplinary}, month = 12, pages = {857-864}, title = {Towards a Mobile Robot Localization Benchmark with Challenging Sensordata in an Industrial Environment}, year = 2021 }
• 
  Towards Splitting Monolithic Workflows into Serverless Functions and Estimating Their Run-Time in the Earth Observation Domain. D. Kaiser; B. Dovhan; A. Bauer; S. Kounev; in Proceedings of the 12th Symposium on Software Performance (SSP) (2021).

  Extended Abstract
  • [ BibTeX ]
  @inproceedings{kaiser2021split, author = {Kaiser, Dennis and Dovhan, Bohdan and Bauer, Andr{\'e} and Kounev, Samuel}, booktitle = {Proceedings of the 12th Symposium on Software Performance (SSP)}, keywords = {severless}, month = 11, note = {Extended Abstract}, title = {{Towards Splitting Monolithic Workflows into Serverless Functions and Estimating Their Run-Time in the Earth Observation Domain}}, year = 2021 }
• 
  People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot. F. Spiess; L. Reinhart; N. Strobel; D. Kaiser; S. Kounev; T. Kaupp; in Journal of Image and Graphics, (J. of Image; Graphics, eds.) (2021). 9(4) 135–139.
  • [ BibTeX ]
  • [ URL ]
  @article{spiess2021people, author = {Spiess, Florian and Reinhart, Lucas and Strobel, Norbert and Kaiser, Dennis and Kounev, Samuel and Kaupp, Tobias}, editor = {of Image, Journal and Graphics}, journal = {Journal of Image and Graphics}, keywords = {t_journalmagazine}, month = 12, number = 4, pages = {135-139}, title = {People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot}, volume = 9, year = 2021 }
• 
  Measuring the Performance Impact of Branching Instructions. L. Beierlieb; L. Iffländer; A. Milenkoski; T. Prantl; S. Kounev; in Proceedings of the 12th Symposium on Software Performance 2021 (SSP’21) (2021).
  • [ BibTeX ]
  @inproceedings{Beierlieb_2021_MeasuringthePerformanceImpactofBranchingInstructions., author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Prantl, Thomas and Kounev, Samuel}, booktitle = {Proceedings of the 12th Symposium on Software Performance 2021 (SSP'21)}, keywords = {sys:relevantfor:se-group}, month = 11, title = {Measuring the Performance Impact of Branching Instructions}, year = 2021 }
• 
  Security and Privacy Aspects of Digital Contact Tracing. F. Roos; Thesis; University of Würzburg. (2021, October).
  • [ BibTeX ]
  @mastersthesis{roos2021security, author = {Roos, Filipp}, keywords = {security}, month = 10, school = {University of W{\"u}rzburg}, title = {Security and Privacy Aspects of Digital Contact Tracing}, type = {Master Thesis}, year = 2021 }
• 
  A Simulation-based Optimization Framework for Online Adaptation of Networks. S. Herrnleben; J. Grohmann; P. Rygielski; V. Lesch; C. Krupitzer; S. Kounev; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools), H. Song, D. Jiang (eds.) (2021). 513–532.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.

  @inproceedings{HeGrRyLeKrKo2020-SIMUtools-Network-Online-Adaptation, abstract = {Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.}, address = {Cham}, author = {Herrnleben, Stefan and Grohmann, Johannes and Rygielski, Pitor and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, editor = {Song, Houbing and Jiang, Dingde}, keywords = {networking}, month = {08}, pages = {513–532}, publisher = {Springer International Publishing}, series = {SIMUtools 2020}, title = {A Simulation-based Optimization Framework for Online Adaptation of Networks}, year = 2021 }
• 
  A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0. M. Züfle; J. Agne; J. Grohmann; I. Dörtoluk; S. Kounev; in Proceedings of the 21st IEEE IES International Conference on Industrial Informatics (2021).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.

  @inproceedings{ZuAgGrDoKo-INDIN2021-PdM, abstract = {Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.}, author = {Züfle, Marwin and Agne, Joachim and Grohmann, Johannes and Dörtoluk, Ibrahim and Kounev, Samuel}, booktitle = {Proceedings of the 21st IEEE IES International Conference on Industrial Informatics}, keywords = {t_interdisciplinary}, month = {07}, publisher = {IEEE}, series = {INDIN'21}, title = {A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0}, year = 2021 }
• 
  SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation. J. Grohmann; S. Eismann; A. Bauer; S. Spinner; J. Blum; N. Herbst; S. Kounev; in ACM Transactions on Autonomous and Adaptive Systems (2021). 15(2)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.

  @article{GrEiBaSpBlHeKo2021-TaaS-SARDE, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andr\'{e} and Spinner, Simon and Blum, Johannes and Herbst, Nikolas and Kounev, Samuel}, journal = {ACM Transactions on Autonomous and Adaptive Systems}, keywords = {t_journalmagazine}, month = {06}, number = 2, publisher = {Association for Computing Machinery}, title = {SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation}, volume = 15, year = 2021 }
• 
  Towards Scientific and Interoperable Earth Observation Exploitation Platforms. J. Eberle; M. Schwinger; H. Zwenzner; in Proceedings of the 2021 conference on Big Data from Space, P. Soille, S. Loekken, M. Albani (eds.) (2021). 89–92.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Earth Observation (EO) exploitation platforms allow users to bring their algorithms and processing components to infrastructures where big EO data collections and compute resources are available. Working on different platforms, interoperability of these needs to be ensured - not only relevant for service specifications, but also for the contents of data inventories and metadata. The interactive development of algorithms and the possibility to share code and data with other users are further feedbacks from scientific users. Additionally, data processing needs to be reproducible and transferable to multiple EO exploitation platforms. In this paper, existing technologies and specifications for data discovery, access, and processing have been analyzed with focus on their usage in two connected cloud infrastructures and related to initial user feedbacks collected from scientists.

  @inproceedings{dlr145824, abstract = {Earth Observation (EO) exploitation platforms allow users to bring their algorithms and processing components to infrastructures where big EO data collections and compute resources are available. Working on different platforms, interoperability of these needs to be ensured - not only relevant for service specifications, but also for the contents of data inventories and metadata. The interactive development of algorithms and the possibility to share code and data with other users are further feedbacks from scientific users. Additionally, data processing needs to be reproducible and transferable to multiple EO exploitation platforms. In this paper, existing technologies and specifications for data discovery, access, and processing have been analyzed with focus on their usage in two connected cloud infrastructures and related to initial user feedbacks collected from scientists.}, author = {Eberle, Jonas and Schwinger, Maximilian and Zwenzner, Hendrik}, booktitle = {Proceedings of the 2021 conference on Big Data from Space}, editor = {Soille, Pierre and Loekken, Sveinung and Albani, Mirko}, keywords = {t_interdisciplinary}, month = {05}, pages = {89–92}, publisher = {Publications Office of the European Union}, title = {Towards Scientific and Interoperable Earth Observation Exploitation Platforms}, year = 2021 }
• 
  Scalable Processing Of Copernicus Sentinel satellite Images Using Argo Workflows. F. W. Fichtner; N. Mandery; M. Schwinger; J. Eberle; M. Nolde; T. Riedlinger; in Big Data from Space 2021 (2021). 77–80.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  With the goal of using a pool of resources dynamically for different applications we migrated scientific Earth Observation processors from dedicated Virtual Machines into a centralized cloud. Processing in this environment works using off-the-shelf components like Kubernetes, Docker and Argo Workflows. We were able to benefit from sharing data, tools and infrastructure in between processors. To showcase the new workflow or pipeline by means of an example we use our burnt area monitoring processor based on Sentinel-3 satellite data. A new architecture based on existing components was established, which we plan to extend it in the future.

  @inproceedings{dlr142309, abstract = {With the goal of using a pool of resources dynamically for different applications we migrated scientific Earth Observation processors from dedicated Virtual Machines into a centralized cloud. Processing in this environment works using off-the-shelf components like Kubernetes, Docker and Argo Workflows. We were able to benefit from sharing data, tools and infrastructure in between processors. To showcase the new workflow or pipeline by means of an example we use our burnt area monitoring processor based on Sentinel-3 satellite data. A new architecture based on existing components was established, which we plan to extend it in the future.}, author = {Fichtner, Florian Willy and Mandery, Nico and Schwinger, Maximilian and Eberle, Jonas and Nolde, Michael and Riedlinger, Torsten}, booktitle = {Big Data from Space 2021}, keywords = {t_interdisciplinary}, month = {05}, pages = {77–80}, title = {Scalable Processing Of Copernicus Sentinel satellite Images Using Argo Workflows}, year = 2021 }
• 
  Libra: A Benchmark for Time Series Forecasting Methods. A. Bauer; M. Züfle; S. Eismann; J. Grohmann; N. Herbst; S. Kounev; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.

  @inproceedings{bauer2021benchmark, abstract = {In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.}, address = {New York, NY, USA}, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {benchmark}, month = {04}, organization = {ACM}, title = {Libra: A Benchmark for Time Series Forecasting Methods}, year = 2021 }
• 
  Energy-Efficiency Comparison of Common Sorting Algorithms. N. Schmitt; S. Kamthania; N. Rawtani; L. Mendoza; K.-D. Lange; S. Kounev; in 2021 29th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2021). 1–8.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  With the rising demand for information technology comes an increase in energy consumption to power it. Namely, cloud computing is constantly growing, in part due to a rising number of devices using the cloud to provide certain functionality. This growth leads to an increase in energy consumption in data centers and is estimated to climb to over 1PWh in 2030. Hardware manufacturers counter the rising demand for energy in cloud data centers by providing techniques to make servers more energy-efficient. However, the advances cannot fully compensate for the growth. To further increase energy efficiency, software needs to be addressed as well but is often neglected by the developers.In this paper, we compare six sorting algorithms, a common task in most programs, against each other in terms of energy efficiency to allow developers to select the best solution to their problem. We selected well-known algorithms in two variants, and two implementation languages, C and Python. We ran each algorithm on two, out of four, state-of-the-art server systems with different CPUs.

  @inproceedings{9614299, abstract = {With the rising demand for information technology comes an increase in energy consumption to power it. Namely, cloud computing is constantly growing, in part due to a rising number of devices using the cloud to provide certain functionality. This growth leads to an increase in energy consumption in data centers and is estimated to climb to over 1PWh in 2030. Hardware manufacturers counter the rising demand for energy in cloud data centers by providing techniques to make servers more energy-efficient. However, the advances cannot fully compensate for the growth. To further increase energy efficiency, software needs to be addressed as well but is often neglected by the developers.In this paper, we compare six sorting algorithms, a common task in most programs, against each other in terms of energy efficiency to allow developers to select the best solution to their problem. We selected well-known algorithms in two variants, and two implementation languages, C and Python. We ran each algorithm on two, out of four, state-of-the-art server systems with different CPUs.}, author = {Schmitt, Norbert and Kamthania, Supriya and Rawtani, Nishant and Mendoza, Luis and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {2021 29th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {power}, month = 11, pages = {1-8}, title = {Energy-Efficiency Comparison of Common Sorting Algorithms}, year = 2021 }
• 
  Towards a Group Encryption Scheme Benchmark: A View on Centralized Schemes with focus on IoT. T. Prantl; P. Ten; L. Iffländer; S. Herrnleben; A. Dmitrenko; S. Kounev; C. Krupitzer; in 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
  • [ BibTeX ]
  @inproceedings{PrTeIfHeDmKo2021-ICPE-GroupEncryptionCentralizedBenchmark, author = {Prantl, Thomas and Ten, Peter and Iffländer, Lukas and Herrnleben, Stefan and Dmitrenko, Alexandra and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2021 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {Group\_Encyrption\_Scheme}, month = {04}, series = {ICPE'21}, title = {Towards a Group Encryption Scheme Benchmark: A View on Centralized Schemes with focus on IoT}, year = 2021 }
• 
  Attack-aware Security Function Management. L. Iffländer; Thesis; Universität Würzburg. (2021, January).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Over the last decades, cybersecurity has become an increasingly important issue. Between 2019 and 2011 alone, the losses from cyberattacks in the United States grew by 6217%. At the same time, attacks became not only more intensive but also more and more versatile and diverse. Cybersecurity has become everyone’s concern. Today, service providers require sophisticated and extensive security infrastructures comprising many security functions dedicated to various cyberattacks. Still, attacks become more violent to a level where infrastructures can no longer keep up. Simply scaling up is no longer sufficient. To address this challenge, in a whitepaper, the Cloud Security Alliance (CSA) proposed multiple work packages for security infrastructure, leveraging the possibilities of Software-defined Networking (SDN) and Network Function Virtualization (NFV). Security functions require a more sophisticated modeling approach than regular network functions. Notably, the property to drop packets deemed malicious has a significant impact on Security Service Function Chains (SSFCs)—service chains consisting of multiple security functions to protect against multiple at- tack vectors. Under attack, the order of these chains influences the end-to-end system performance depending on the attack type. Unfortunately, it is hard to predict the attack composition at system design time. Thus, we make a case for dynamic attack-aware SSFC reordering. Also, we tackle the issues of the lack of integration between security functions and the surrounding network infrastructure, the insufficient use of short term CPU frequency boosting, and the lack of Intrusion Detection and Prevention Systems (IDPS) against database ransomware attacks. Current works focus on characterizing the performance of security functions and their behavior under overload without considering the surrounding infrastructure. Other works aim at replacing security functions using network infrastructure features but do not consider integrating security functions within the network. Further publications deal with using SDN for security or how to deal with new vulnerabilities introduced through SDN. However, they do not take security function performance into account. NFV is a popular field for research dealing with frameworks, benchmarking methods, the combination with SDN, and implementing security functions as Virtualized Network Functions (VNFs). Research in this area brought forth the concept of Service Function Chains (SFCs) that chain multiple network functions after one another. Nevertheless, they still do not consider the specifics of security functions. The mentioned CSA whitepaper proposes many valuable ideas but leaves their realization open to others. This thesis presents solutions to increase the performance of single security functions using SDN, performance modeling, a framework for attack-aware SSFC reordering, a solution to make better use of CPU frequency boosting, and an IDPS against database ransomware. Specifically, the primary contributions of this work are: • We present approaches to dynamically bypass Intrusion Detection Systems (IDS) in order to increase their performance without reducing the security level. To this end, we develop and implement three SDN-based approaches (two dynamic and one static). We evaluate the proposed approaches regarding security and performance and show that they significantly increase the performance com- pared to an inline IDS without significant security deficits. We show that using software switches can further increase the performance of the dynamic approaches up to a point where they can eliminate any throughput drawbacks when using the IDS. • We design a DDoS Protection System (DPS) against TCP SYN flood at tacks in the form of a VNF that works inside an SDN-enabled network. This solution eliminates known scalability and performance drawbacks of existing solutions for this attack type. Then, we evaluate this solution showing that it correctly handles the connection establishment and present solutions for an observed issue. Next, we evaluate the performance showing that our solution increases performance up to three times. Parallelization and parameter tuning yields another 76% performance boost. Based on these findings, we discuss optimal deployment strategies. • We introduce the idea of attack-aware SSFC reordering and explain its impact in a theoretical scenario. Then, we discuss the required information to perform this process. We validate our claim of the importance of the SSFC order by analyzing the behavior of single security functions and SSFCs. Based on the results, we conclude that there is a massive impact on the performance up to three orders of magnitude, and we find contradicting optimal orders for different workloads. Thus, we demonstrate the need for dynamic reordering. Last, we develop a model for SSFC regarding traffic composition and resource demands. We classify the traffic into multiple classes and model the effect of single security functions on the traffic and their generated resource demands as functions of the incoming network traffic. Based on our model, we propose three approaches to determine optimal orders for reordering. • We implement a framework for attack-aware SSFC reordering based on this knowledge. The framework places all security functions inside an SDN-enabled network and reorders them using SDN flows. Our evaluation shows that the framework can enforce all routes as desired. It correctly adapts to all attacks and returns to the original state after the attacks cease. We find possible security issues at the moment of reordering and present solutions to eliminate them. • Next, we design and implement an approach to load balance servers while taking into account their ability to go into a state of Central Processing Unit (CPU) frequency boost. To this end, the approach collects temperature information from available hosts and places services on the host that can attain the boosted mode the longest. We evaluate this approach and show its effectiveness. For high load scenarios, the approach increases the overall performance and the performance per watt. Even better results show up for low load workloads, where not only all performance metrics improve but also the temperatures and total power consumption decrease. • Last, we design an IDPS protecting against database ransomware attacks that comprise multiple queries to attain their goal. Our solution models these attacks using a Colored Petri Net (CPN). A proof-of-concept implementation shows that our approach is capable of detecting attacks without creating false positives for benign scenarios. Furthermore, our solution creates only a small performance impact. Our contributions can help to improve the performance of security infrastructures. We see multiple application areas from data center operators over software and hardware developers to security and performance researchers. Most of the above-listed contributions found use in several research publications. Regarding future work, we see the need to better integrate SDN-enabled security functions and SSFC reordering in data center networks. Future SSFC should discriminate between different traffic types, and security frameworks should support automatically learning models for security functions. We see the need to consider energy efficiency when regarding SSFCs and take CPU boosting technologies into account when designing performance models as well as placement, scaling, and deployment strategies. Last, for a faster adaptation against recent ransomware attacks, we propose machine-assisted learning for database IDPS signatures.

  @phdthesis{ifflander2021attackaware, abstract = {Over the last decades, cybersecurity has become an increasingly important issue. Between 2019 and 2011 alone, the losses from cyberattacks in the United States grew by 6217%. At the same time, attacks became not only more intensive but also more and more versatile and diverse. Cybersecurity has become everyone’s concern. Today, service providers require sophisticated and extensive security infrastructures comprising many security functions dedicated to various cyberattacks. Still, attacks become more violent to a level where infrastructures can no longer keep up. Simply scaling up is no longer sufficient. To address this challenge, in a whitepaper, the Cloud Security Alliance (CSA) proposed multiple work packages for security infrastructure, leveraging the possibilities of Software-defined Networking (SDN) and Network Function Virtualization (NFV). Security functions require a more sophisticated modeling approach than regular network functions. Notably, the property to drop packets deemed malicious has a significant impact on Security Service Function Chains (SSFCs)—service chains consisting of multiple security functions to protect against multiple at- tack vectors. Under attack, the order of these chains influences the end-to-end system performance depending on the attack type. Unfortunately, it is hard to predict the attack composition at system design time. Thus, we make a case for dynamic attack-aware SSFC reordering. Also, we tackle the issues of the lack of integration between security functions and the surrounding network infrastructure, the insufficient use of short term CPU frequency boosting, and the lack of Intrusion Detection and Prevention Systems (IDPS) against database ransomware attacks. Current works focus on characterizing the performance of security functions and their behavior under overload without considering the surrounding infrastructure. Other works aim at replacing security functions using network infrastructure features but do not consider integrating security functions within the network. Further publications deal with using SDN for security or how to deal with new vulnerabilities introduced through SDN. However, they do not take security function performance into account. NFV is a popular field for research dealing with frameworks, benchmarking methods, the combination with SDN, and implementing security functions as Virtualized Network Functions (VNFs). Research in this area brought forth the concept of Service Function Chains (SFCs) that chain multiple network functions after one another. Nevertheless, they still do not consider the specifics of security functions. The mentioned CSA whitepaper proposes many valuable ideas but leaves their realization open to others. This thesis presents solutions to increase the performance of single security functions using SDN, performance modeling, a framework for attack-aware SSFC reordering, a solution to make better use of CPU frequency boosting, and an IDPS against database ransomware. Specifically, the primary contributions of this work are: • We present approaches to dynamically bypass Intrusion Detection Systems (IDS) in order to increase their performance without reducing the security level. To this end, we develop and implement three SDN-based approaches (two dynamic and one static). We evaluate the proposed approaches regarding security and performance and show that they significantly increase the performance com- pared to an inline IDS without significant security deficits. We show that using software switches can further increase the performance of the dynamic approaches up to a point where they can eliminate any throughput drawbacks when using the IDS. • We design a DDoS Protection System (DPS) against TCP SYN flood at tacks in the form of a VNF that works inside an SDN-enabled network. This solution eliminates known scalability and performance drawbacks of existing solutions for this attack type. Then, we evaluate this solution showing that it correctly handles the connection establishment and present solutions for an observed issue. Next, we evaluate the performance showing that our solution increases performance up to three times. Parallelization and parameter tuning yields another 76% performance boost. Based on these findings, we discuss optimal deployment strategies. • We introduce the idea of attack-aware SSFC reordering and explain its impact in a theoretical scenario. Then, we discuss the required information to perform this process. We validate our claim of the importance of the SSFC order by analyzing the behavior of single security functions and SSFCs. Based on the results, we conclude that there is a massive impact on the performance up to three orders of magnitude, and we find contradicting optimal orders for different workloads. Thus, we demonstrate the need for dynamic reordering. Last, we develop a model for SSFC regarding traffic composition and resource demands. We classify the traffic into multiple classes and model the effect of single security functions on the traffic and their generated resource demands as functions of the incoming network traffic. Based on our model, we propose three approaches to determine optimal orders for reordering. • We implement a framework for attack-aware SSFC reordering based on this knowledge. The framework places all security functions inside an SDN-enabled network and reorders them using SDN flows. Our evaluation shows that the framework can enforce all routes as desired. It correctly adapts to all attacks and returns to the original state after the attacks cease. We find possible security issues at the moment of reordering and present solutions to eliminate them. • Next, we design and implement an approach to load balance servers while taking into account their ability to go into a state of Central Processing Unit (CPU) frequency boost. To this end, the approach collects temperature information from available hosts and places services on the host that can attain the boosted mode the longest. We evaluate this approach and show its effectiveness. For high load scenarios, the approach increases the overall performance and the performance per watt. Even better results show up for low load workloads, where not only all performance metrics improve but also the temperatures and total power consumption decrease. • Last, we design an IDPS protecting against database ransomware attacks that comprise multiple queries to attain their goal. Our solution models these attacks using a Colored Petri Net (CPN). A proof-of-concept implementation shows that our approach is capable of detecting attacks without creating false positives for benign scenarios. Furthermore, our solution creates only a small performance impact. Our contributions can help to improve the performance of security infrastructures. We see multiple application areas from data center operators over software and hardware developers to security and performance researchers. Most of the above-listed contributions found use in several research publications. Regarding future work, we see the need to better integrate SDN-enabled security functions and SSFC reordering in data center networks. Future SSFC should discriminate between different traffic types, and security frameworks should support automatically learning models for security functions. We see the need to consider energy efficiency when regarding SSFCs and take CPU boosting technologies into account when designing performance models as well as placement, scaling, and deployment strategies. Last, for a faster adaptation against recent ransomware attacks, we propose machine-assisted learning for database IDPS signatures.}, author = {Iffl{\"a}nder, Lukas}, keywords = {security}, month = {01}, school = {Universit{\"a}t W{\"u}rzburg}, title = {Attack-aware Security Function Management}, type = {doctoralthesis}, year = 2021 }
• 
  Performance Impact Analysis of Securing MQTT Using TLS. T. Prantl; L. Iffländer; S. Herrnleben; S. Engel; S. Kounev; C. Krupitzer; in 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
  • [ BibTeX ]
  @inproceedings{PrTeIfHeDmKo2021-ICPE-MQTT_TLS, author = {Prantl, Thomas and Iffländer, Lukas and Herrnleben, Stefan and Engel, Simon and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2021 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {IoT}, month = {04}, series = {ICPE'21}, title = {Performance Impact Analysis of Securing MQTT Using TLS}, year = 2021 }
• 
  Performance Evaluation of a Post-Quantum Public-Key Cryptosystem. T. Prantl; D. Prantl; L. Beierlieb; L. Iffländer; A. Dmitrienko; C. Krupitzer; S. Kounev; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
  • [ BibTeX ]
  @inproceedings{PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem, author = {Prantl, Thomas and Prantl, Dominik and Beierlieb, Lukas and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {t_workshop}, organization = {IEEE}, title = {Performance Evaluation of a Post-Quantum Public-Key Cryptosystem}, year = 2021 }
• 
  Vision: Challenges & Opportunities. M. Annaaram; N. Asokan; B. G. Atli; S. Avestimeh; F. Brasser; R. Cammarota; A. Dmitrienko; A. Dziedzic; T. Given-Wilson; L. J. Gunn; F. Kerschbaum; F. Koushanfar; A. Legay; M. Miettinen; T. D. Nguyen; N. Papernot; A.-R. Sadeghi; M. Schunter; R. Shokri; V. Smith; in Vision Paper of the Private AI Collaborative Research Institute (2021).
  • [ BibTeX ]
  @inproceedings{annaaramvision, author = {Annaaram, Murali and Asokan, N. and Atli, Buse Gül and Avestimeh, Salman and Brasser, Ferdinand and Cammarota, Rosario and Dmitrienko, Alexandra and Dziedzic, Adam and Given-Wilson, Thomas and Gunn, Lachlan J. and Kerschbaum, Florian and Koushanfar, Farinaz and Legay, Axel and Miettinen, Markus and Nguyen, Thien Duc and Papernot, Nicolas and Sadeghi, Ahmad-Reza and Schunter, Matthias and Shokri, Reza and Smith, Virginia}, booktitle = {Vision Paper of the Private AI Collaborative Research Institute}, keywords = {sss-group}, title = {Vision: Challenges & Opportunities.}, year = 2021 }
• 
  Sizeless: Predicting the Optimal Size of Serverless Functions. S. Eismann; L. Bui; J. Grohmann; C. Abad; N. Herbst; S. Kounev; in Proceedings of the 22nd International MIDDLEWARE Conference (2021). 248–259.

  Best Student Paper Award, ACM Artifacts Evaluated — Functional
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.

  @inproceedings{eismann2021sizeless, abstract = {Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.}, author = {Eismann, Simon and Bui, Long and Grohmann, Johannes and Abad, Cristina and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 22nd International MIDDLEWARE Conference}, keywords = {descartes}, note = {Best Student Paper Award, ACM Artifacts Evaluated — Functional}, pages = {248–259}, title = {Sizeless: Predicting the Optimal Size of Serverless Functions}, year = 2021 }
• 
  Serverless Applications:Why, When, and How?. S. Eismann; S. Joel; E. van Eyk; M. Schwinger; J. Grohmann; N. Herbst; C. Abad; A. Iosup; in IEEE Software (2021). 38(1) 32–39.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Serverless computing shows good promise for efficiency and ease-of-use. Yet, there are only a few, scattered and sometimes conflicting reports on questions such as Why do so many companies adopt serverless?, When are serverless applications well suited?, and How are serverless applications currently implemented? To address these questions, we analyze 89 serverless applications from open-source projects, industrial sources, academic literature, and scientific computing—the most extensive study to date.

  @article{eismann2020serverless, abstract = {Serverless computing shows good promise for efficiency and ease-of-use. Yet, there are only a few, scattered and sometimes conflicting reports on questions such as Why do so many companies adopt serverless?, When are serverless applications well suited?, and How are serverless applications currently implemented? To address these questions, we analyze 89 serverless applications from open-source projects, industrial sources, academic literature, and scientific computing—the most extensive study to date.}, author = {Eismann, Simon and Joel, Scheuner and van Eyk, Erwin and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina and Iosup, Alexandru}, journal = {IEEE Software}, keywords = {t_journalmagazine}, number = 1, pages = {32–39}, title = {Serverless Applications:Why, When, and How?}, volume = 38, year = 2021 }
• 
  Remote Attestation for IoT with Smart Verifier. L. Petzi; Thesis; University of Würzburg. (2021, January).
  • [ Abstract ]
  • [ BibTeX ]
  The Internet is evolving. The Internet of Things (IoT) is a disruptive technology, billions of small and smart devices are encompassing every aspect of our lives, which leads us to a new era of the Internet. IoT connects billions of heterogeneous embedded devices in large networks. Restricted capabilities of the devices in combination with various mutually mistrusting parties throughout the network pose new challenges to security. As IoT devices become increasingly important, so does trust in their secure and reliable operation. But before we can trust them, we need to establish the trust. In the past, remote attestation has emerged into a very powerful tool in order to establish trust between devices. Traditional remote attestation establishes a static trust between two devices. This approach suffers badly heterogeneity of IoT network and purely scales to a large number of devices. Furthermore, it is very vulnerable to Denial of Service attacks. Within this thesis, we present a new approach to Remote Attestation in IoT networks. Our design combines the core strength of Remote Attestation with the advantages blockchain technology in order to enable attestation and validation of attestation evidences in a scalable manner. At rst, we develop a remote attestation scheme on a low end device serving as an exemplary IoT device. Second, we design our smart verifier, a flexible system capable of validating and storing attestation evidence. The system is build on top of blockchain technology and solves several issues accompanied by Traditional Remote Attestation in IoT such as scalability and device heterogeneity. In the end, we build and evaluate a proof of concept implementation based on ARM TrustZone and Sawtooth Hyperledger. The proposed system architecture enables trust establishment in a scalable manner even in heterogeneous networks while preventing denial of service attacks

  @mastersthesis{chan2019remote, abstract = {The Internet is evolving. The Internet of Things (IoT) is a disruptive technology, billions of small and smart devices are encompassing every aspect of our lives, which leads us to a new era of the Internet. IoT connects billions of heterogeneous embedded devices in large networks. Restricted capabilities of the devices in combination with various mutually mistrusting parties throughout the network pose new challenges to security. As IoT devices become increasingly important, so does trust in their secure and reliable operation. But before we can trust them, we need to establish the trust. In the past, remote attestation has emerged into a very powerful tool in order to establish trust between devices. Traditional remote attestation establishes a static trust between two devices. This approach suffers badly heterogeneity of IoT network and purely scales to a large number of devices. Furthermore, it is very vulnerable to Denial of Service attacks. Within this thesis, we present a new approach to Remote Attestation in IoT networks. Our design combines the core strength of Remote Attestation with the advantages blockchain technology in order to enable attestation and validation of attestation evidences in a scalable manner. At rst, we develop a remote attestation scheme on a low end device serving as an exemplary IoT device. Second, we design our smart verifier, a flexible system capable of validating and storing attestation evidence. The system is build on top of blockchain technology and solves several issues accompanied by Traditional Remote Attestation in IoT such as scalability and device heterogeneity. In the end, we build and evaluate a proof of concept implementation based on ARM TrustZone and Sawtooth Hyperledger. The proposed system architecture enables trust establishment in a scalable manner even in heterogeneous networks while preventing denial of service attacks}, author = {Petzi, Lukas}, booktitle = {CVPR Workshops}, keywords = {security}, month = {01}, school = {University of W{\"u}rzburg}, title = {Remote Attestation for IoT with Smart Verifier}, type = {Bachelor Thesis}, year = 2021 }
• 
  SPEC Research Group Newsletter, vol. 3 no. 2. S. Kounev; A. van Hoorn; A. Bauer; (2021, April).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-10, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{\'e}}, keywords = {SPEC}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 3 no. 2}}, year = 2021 }
• 
  Utility-based Vehicle Routing Integrating User Preferences. V. Lesch; M. Hadry; S. Kounev; C. Krupitzer; in Proceedings of 3rd International Workshop on Pervasive Computing for Vehicular Systems (PerVehicle), 2021 (2021).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{lesch2021utility, author = {Lesch, Veronika and Hadry, Marius and Kounev, Samuel and Krupitzer, Christian}, booktitle = {Proceedings of 3rd International Workshop on Pervasive Computing for Vehicular Systems (PerVehicle), 2021}, keywords = {t_workshop}, month = {03}, publisher = {IEEE}, title = {Utility-based Vehicle Routing Integrating User Preferences}, year = 2021 }
• 
  ESCORT: Ethereum Smart COntRacTs Vulnerability Detection using Deep Neural Network and Transfer Learning. O. Lutz; H. Chen; H. Fereidooni; C. Sendner; A. Dmitrienko; A. R. Sadeghi; F. Koushanfar; in ArXiv | arXiv:2103.12607v1 (2021).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Ethereum smart contracts are automated decentralized applications on the blockchain that describe the terms of the agreement between buyers and sellers, reducing the need for trusted intermediaries and arbitration. However, the deployment of smart contracts introduces new attack vectors into the cryptocurrency systems. In particular, programming flaws in smart contracts can be and have already been exploited to gain enormous financial profits. It is thus an emerging yet crucial issue to detect vulnerabilities of different classes in contracts in an efficient manner. Existing machine learning-based vulnerability detection methods are limited and only inspect whether the smart contract is vulnerable, or train individual classifiers for each specific vulnerability, or demonstrate multi-class vulnerability detection without extensibility consideration. To overcome the scalability and generalization limitations of existing works, we propose ESCORT, the first Deep Neural Network (DNN)-based vulnerability detection framework for Ethereum smart contracts that support lightweight transfer learning on unseen security vulnerabilities, thus is extensible and generalizable. ESCORT leverages a multi-output NN architecture that consists of two parts: (i) A common feature extractor that learns the semantics of the input contract; (ii) Multiple branch structures where each branch learns a specific vulnerability type based on features obtained from the feature extractor. Experimental results show that ESCORT achieves an average F1-score of 95% on six vulnerability types and the detection time is 0.02 seconds per contract. When extended to new vulnerability types, ESCORT yields an average F1-score of 93%. To the best of our knowledge, ESCORT is the first framework that enables transfer learning on new vulnerability types with minimal modification of the DNN model architecture and re-training overhead.

  @article{lutz2021escort, abstract = {Ethereum smart contracts are automated decentralized applications on the blockchain that describe the terms of the agreement between buyers and sellers, reducing the need for trusted intermediaries and arbitration. However, the deployment of smart contracts introduces new attack vectors into the cryptocurrency systems. In particular, programming flaws in smart contracts can be and have already been exploited to gain enormous financial profits. It is thus an emerging yet crucial issue to detect vulnerabilities of different classes in contracts in an efficient manner. Existing machine learning-based vulnerability detection methods are limited and only inspect whether the smart contract is vulnerable, or train individual classifiers for each specific vulnerability, or demonstrate multi-class vulnerability detection without extensibility consideration. To overcome the scalability and generalization limitations of existing works, we propose ESCORT, the first Deep Neural Network (DNN)-based vulnerability detection framework for Ethereum smart contracts that support lightweight transfer learning on unseen security vulnerabilities, thus is extensible and generalizable. ESCORT leverages a multi-output NN architecture that consists of two parts: (i) A common feature extractor that learns the semantics of the input contract; (ii) Multiple branch structures where each branch learns a specific vulnerability type based on features obtained from the feature extractor. Experimental results show that ESCORT achieves an average F1-score of 95% on six vulnerability types and the detection time is 0.02 seconds per contract. When extended to new vulnerability types, ESCORT yields an average F1-score of 93%. To the best of our knowledge, ESCORT is the first framework that enables transfer learning on new vulnerability types with minimal modification of the DNN model architecture and re-training overhead.}, author = {Lutz, Oliver and Chen, Huili and Fereidooni, Hossein and Sendner, Christoph and Dmitrienko, Alexandra and Sadeghi, Ahmad Reza and Koushanfar, Farinaz}, journal = {ArXiv | arXiv:2103.12607v1}, keywords = {cryptography}, month = {03}, title = {ESCORT: Ethereum Smart COntRacTs Vulnerability Detection using Deep Neural Network and Transfer Learning}, year = 2021 }
• 
  SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications. J. Grohmann; M. Straesser; A. Chalbani; S. Eismann; Y. Arian; N. Herbst; N. Peretz; S. Kounev; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).

  Acceptance Rate: 29%
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing---a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90%.

  @inproceedings{GrStChEiArHePeKo2021-ICPE, abstract = {Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing—a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90\%.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Straesser, Martin and Chalbani, Avi and Eismann, Simon and Arian, Yair and Herbst, Nikolas and Peretz, Noam and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {performance}, month = {04}, note = {Acceptance Rate: 29%}, organization = {ACM}, title = {SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications}, year = 2021 }
• 
  A Comparison of Mechanisms for Compensating Negative Impacts of System Integration. V. Lesch; C. Krupitzer; K. Stubenrauch; N. Keil; C. Becker; S. Kounev; M. Segata; in Future Generation Computer Systems (2021). 116 117–131.
  • [ BibTeX ]
  • [ DOI ]
  • [ Download ]
  @article{lesch2020comparison, author = {Lesch, Veronika and Krupitzer, Christian and Stubenrauch, Kevin and Keil, Nico and Becker, Christian and Kounev, Samuel and Segata, Michele}, journal = {Future Generation Computer Systems}, keywords = {platooning}, month = {03}, pages = {117-131}, title = {A Comparison of Mechanisms for Compensating Negative Impacts of System Integration}, volume = 116, year = 2021 }

• 
  Systems Benchmarking S. Kounev; K.-D. Lange; J. von Kistowski; (2020). (1st ed.) Springer International Publishing.
  • [ BibTeX ]
  @book{KoLaKi-2020-SystemsBenchmarking, author = {Kounev, Samuel and Lange, Klaus-Dieter and von Kistowski, Jóakim}, edition = 1, keywords = {t_book}, publisher = {Springer International Publishing}, title = {{Systems Benchmarking}}, year = 2020 }
• 
  Beyond Microbenchmarks: The SPEC-RG Vision for A Comprehensive Serverless Benchmark. E. van Eyk; J. Scheuner; S. Eismann; C. Abad; A. Iosup; in Companion of the 2020 ACM/SPEC International Conference on Performance Engineering (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Serverless computing services, such as Function-as-a-Service (FaaS), hold the attractive promise of a high level of abstraction and high performance, combined with the minimization of operational logic. Several large ecosystems of serverless platforms, both open- and closed-source, aim to realize this promise. Consequently, a lucrative market has emerged. However, the performance trade-offs of these systems are not well-understood. Moreover, it is exactly the high level of abstraction and the opaqueness of the operational-side that make performance evaluation studies of serverless platforms challenging. Learning from the history of IT platforms, we argue that a benchmark for serverless platforms could help address this challenge. We envision a comprehensive serverless benchmark, which we contrast to the narrow focus of prior work in this area. We argue that a comprehensive benchmark will need to take into account more than just runtime overhead, and include notions of cost, realistic workloads, more (open-source) platforms, and cloud integrations. Finally, we show through preliminary real-world experiments how such a benchmark can help compare the performance overhead when running a serverless workload on state-of-the-art platforms.

  @inproceedings{10.1145/3185768.3186293, abstract = {Serverless computing services, such as Function-as-a-Service (FaaS), hold the attractive promise of a high level of abstraction and high performance, combined with the minimization of operational logic. Several large ecosystems of serverless platforms, both open- and closed-source, aim to realize this promise. Consequently, a lucrative market has emerged. However, the performance trade-offs of these systems are not well-understood. Moreover, it is exactly the high level of abstraction and the opaqueness of the operational-side that make performance evaluation studies of serverless platforms challenging. Learning from the history of IT platforms, we argue that a benchmark for serverless platforms could help address this challenge. We envision a comprehensive serverless benchmark, which we contrast to the narrow focus of prior work in this area. We argue that a comprehensive benchmark will need to take into account more than just runtime overhead, and include notions of cost, realistic workloads, more (open-source) platforms, and cloud integrations. Finally, we show through preliminary real-world experiments how such a benchmark can help compare the performance overhead when running a serverless workload on state-of-the-art platforms.}, author = {van Eyk, Erwin and Scheuner, Joel and Eismann, Simon and Abad, Cristina and Iosup, Alexandru}, booktitle = {Companion of the 2020 ACM/SPEC International Conference on Performance Engineering}, keywords = {SPEC}, title = {{Beyond Microbenchmarks: The SPEC-RG Vision for A Comprehensive Serverless Benchmark}}, year = 2020 }
• 
  A Taxonomy of Techniques for SLO Failure Prediction in Software Systems. J. Grohmann; N. Herbst; A. Chalbani; Y. Arian; N. Peretz; S. Kounev; in Computers (2020). 9(1) 10.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.

  @article{grohmann2020-MDPI-taxonomy, abstract = {Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.}, author = {Grohmann, Johannes and Herbst, Nikolas and Chalbani, Avi and Arian, Yair and Peretz, Noam and Kounev, Samuel}, journal = {Computers}, keywords = {automated_model_learning}, number = 1, pages = 10, publisher = {Multidisciplinary Digital Publishing Institute (MDPI)}, title = {A Taxonomy of Techniques for SLO Failure Prediction in Software Systems}, volume = 9, year = 2020 }
• 
  Elasticity of Cloud Platforms. N. Herbst; A. Bauer; S. Kounev; in Systems Benchmarking: For Scientists and Engineers, S. Kounev, K.-D. Lange, J. von Kistowski (eds.) (2020). 319–340.
  • [ BibTeX ]
  @inproceedings{bauer2020bookelasticity, author = {Herbst, Nikolas and Bauer, Andr{\'e} and Kounev, Samuel}, booktitle = {{Systems Benchmarking: For Scientists and Engineers}}, editor = {Kounev, Samuel and Lange, Klaus-Dieter and von Kistowski, J{\'o}akim}, keywords = {t_bookchapter}, month = {01}, organization = {Springer}, pages = {319-340}, title = {{Elasticity of Cloud Platforms}}, year = 2020 }
• 
  A Survey on Human Machine Interaction in Industry 4.0 C. Krupitzer; S. Müller; V. Lesch; M. Züfle; J. Edinger; A. Lemken; D. Schäfer; S. Kounev; C. Becker; (2020).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @techreport{Krupitzer2020ASurvey, author = {Krupitzer, Christian and M{\"u}ller, Sebastian and Lesch, Veronika and Z{\"u}fle, Marwin and Edinger, Janick and Lemken, Alexander and Sch{\"a}fer, Dominik and Kounev, Samuel and Becker, Christian}, institution = {Universit{\"a}t W{\"u}rzburg and University of Mannheim and ioxp GmbH and Syntax Systems GmbH}, journal = {arXiv}, keywords = {t_interdisciplinary}, month = {02}, title = {{A Survey on Human Machine Interaction in Industry 4.0}}, year = 2020 }
• 
  Towards Self-Aware Multirotor Formations. D. Kaiser; V. Lesch; J. Rothe; M. Strohmeier; F. Spiess; C. Krupitzer; S. Montenegro; S. Kounev; in Computers (2020). 9(7)

  Special Issue on Self-Aware Computing
  • [ BibTeX ]
  • [ Download ]
  @article{Kaiser2020Towards, author = {Kaiser, Dennis and Lesch, Veronika and Rothe, Julian and Strohmeier, Michael and Spiess, Florian and Krupitzer, Christian and Montenegro, Sergio and Kounev, Samuel}, journal = {Computers}, keywords = {Intelligent_Transportation_Systems}, month = {02}, note = {Special Issue on Self-Aware Computing}, number = 7, publisher = {MDPI}, title = {{Towards Self-Aware Multirotor Formations}}, volume = 9, year = 2020 }
• 
  A Concept for Crowd-sensed Prediction of Mobile Network Connectivity. S. Herrnleben; B. Zeidler; M. Züfle; C. Krupitzer; S. Kounev; in GI/ITG Workshop on Machine Learning in the Context of Communication Networks 2020 (2020).
  • [ BibTeX ]
  @inproceedings{HeZeZuKrKo-FGMLN2020-Prediction-of-Mobile-Network-Connectivity, author = {Herrnleben, Stefan and Zeidler, Bernd and Z{\"u}fle, Marwin and Krupitzer, Christian and Kounev, Samuel}, booktitle = {GI/ITG Workshop on Machine Learning in the Context of Communication Networks 2020}, keywords = {t_workshop}, month = {02}, title = {{A Concept for Crowd-sensed Prediction of Mobile Network Connectivity}}, year = 2020 }
• 
  A Survey on Predictive Maintenance for Industry 4.0 C. Krupitzer; T. Wagenhals; M. Züfle; V. Lesch; D. Schäfer; A. Mozaffarin; J. Edinger; C. Becker; S. Kounev; (2020).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @techreport{Krupitzer2020ASurveyPdM, author = {Krupitzer, Christian and Wagenhals, Tim and Z{\"u}fle, Marwin and Lesch, Veronika and Sch{\"a}fer, Dominik and Mozaffarin, Amin and Edinger, Janick and Becker, Christian and Kounev, Samuel}, institution = {Universit{\"a}t W{\"u}rzburg and University of Mannheim and Syntax Systems GmbH and MOZYS Engineering GmbH}, journal = {arXiv}, keywords = {Reliability}, month = {02}, title = {{A Survey on Predictive Maintenance for Industry 4.0}}, year = 2020 }
• 
  REACT: A Model-Based Runtime Environment for Adapting Communication Systems. M. Pfannemüller; M. Breitbach; C. Krupitzer; M. Weckesser; C. Becker; B. Schmerl; A. Schürr; in Proceedings of the IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS 2020) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Trends such as the Internet of Things or edge computing lead to a growing number of networked devices. Hence, it is becoming increasingly important to manage communication systems at runtime. Adding self-adaptive capabilities is one approach to reduce administrative effort and cope with changing execution contexts. Existing frameworks for building self-adaptive software can help to reduce development effort in general. Yet, they are neither tailored towards the use in communication systems nor easily usable without profound knowledge in self-adaptive systems development. In this paper, we propose REACT, a reusable, model-based runtime environment to complement communication systems with adaptive behavior. It addresses the heterogeneity and distribution aspects of networks and reduces development effort. REACT empowers developers of communication systems to add adaptive behavior without having experience in self-adaptive systems development. We show the effectiveness and efficiency of our prototype in an experimental evaluation based on two distinct use cases from the communication systems domain: cloud resource management and software-defined networking. The first use case includes a comparison with Rainbow, which represents a state-of-the-art model-based framework for building self-adaptive systems. The second use case applies REACT in a sophisticated, real-world communication system scenario.

  @conference{acsos2020:react, abstract = {Trends such as the Internet of Things or edge computing lead to a growing number of networked devices. Hence, it is becoming increasingly important to manage communication systems at runtime. Adding self-adaptive capabilities is one approach to reduce administrative effort and cope with changing execution contexts. Existing frameworks for building self-adaptive software can help to reduce development effort in general. Yet, they are neither tailored towards the use in communication systems nor easily usable without profound knowledge in self-adaptive systems development. In this paper, we propose REACT, a reusable, model-based runtime environment to complement communication systems with adaptive behavior. It addresses the heterogeneity and distribution aspects of networks and reduces development effort. REACT empowers developers of communication systems to add adaptive behavior without having experience in self-adaptive systems development. We show the effectiveness and efficiency of our prototype in an experimental evaluation based on two distinct use cases from the communication systems domain: cloud resource management and software-defined networking. The first use case includes a comparison with Rainbow, which represents a state-of-the-art model-based framework for building self-adaptive systems. The second use case applies REACT in a sophisticated, real-world communication system scenario.}, author = {Pfannemüller, Martin and Breitbach, Martin and Krupitzer, Christian and Weckesser, Markus and Becker, Christian and Schmerl, Bradley and Schürr, Andy}, booktitle = {Proceedings of the IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS 2020)}, keywords = {self-adaptive-systems}, title = {REACT: A Model-Based Runtime Environment for Adapting Communication Systems}, year = 2020 }
• 
  To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows. M. Züfle; C. Krupitzer; F. Erhard; J. Grohmann; S. Kounev; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020). 19–36.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.

  @inproceedings{ZuKrErGrKo-MMB2020-ToFailOrNotToFail, abstract = {Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.}, address = {Cham}, author = {Z{\"u}fle, Marwin and Krupitzer, Christian and Erhard, Florian and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {Reliability}, month = {03}, pages = {19–36}, publisher = {Springer}, series = {MMB 2020}, title = {{To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows}}, year = 2020 }
• 
  Introduction to the Special Issue ``Applications in Self-Aware Computing Systems and their Evaluation’’. C. Krupitzer; B. Eberhardinger; I. Gerostathopoulos; C. Raibulet; (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The joint 1st Workshop on Evaluations and Measurements in Self-Aware Computing Systems (EMSAC 2019) and Workshop on Self-Aware Computing (SeAC) was held as part of the FAS* conference alliance in conjunction with the 16th IEEE International Conference on Autonomic Computing (ICAC) and the 13th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO) in Umea, Sweden on 20 June 2019. The goal of this one-day workshop was to bring together researchers and practitioners from academic environments and from the industry to share their solutions, ideas, visions, and doubts in self-aware computing systems in general and in the evaluation and measurements of such systems in particular. The workshop aimed to enable discussions, partnerships, and collaborations among the participants. This special issue follows the theme of the workshop. It contains extended versions of workshop presentations as well as additional contributions.

  @misc{KrEbGeRa2020_Computers_seacSI, abstract = {The joint 1st Workshop on Evaluations and Measurements in Self-Aware Computing Systems (EMSAC 2019) and Workshop on Self-Aware Computing (SeAC) was held as part of the FAS* conference alliance in conjunction with the 16th IEEE International Conference on Autonomic Computing (ICAC) and the 13th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO) in Umea, Sweden on 20 June 2019. The goal of this one-day workshop was to bring together researchers and practitioners from academic environments and from the industry to share their solutions, ideas, visions, and doubts in self-aware computing systems in general and in the evaluation and measurements of such systems in particular. The workshop aimed to enable discussions, partnerships, and collaborations among the participants. This special issue follows the theme of the workshop. It contains extended versions of workshop presentations as well as additional contributions.}, author = {Krupitzer, Christian and Eberhardinger, Benedikt and Gerostathopoulos, Ilias and Raibulet, Claudia}, journal = {MDPI Computers}, keywords = {se2}, number = 1, publisher = {MDPI}, title = {{Introduction to the Special Issue ``Applications in Self-Aware Computing Systems and their Evaluation''}}, volume = 9, year = 2020 }
• 
  SIMPL: Secure IoT Management Platform. T. Prantl; A. E. Ben Yahya; A. Dmitrienko; S. Kounev; F. Lipp; D. Hock; C. Rathfelder; M. Hofherr; in ITG Workshop on IT Security (ITSec) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The proliferation of IoT devices is increasing at a fast pace, whether for private or business use. However,despite their growing popularity, their safe operation is often not guaranteed. To tackle the security challenges of modern IoT environments, the German Federal Ministry of Education and Research is funding the SIMPL project in order to support research in this area. The contribution of this paper is to introduce the main goal, objectives, and key features of SIMPL

  @inproceedings{PrYaDmKoLiHoRaHo2020-ITSec-SIMPL, abstract = {The proliferation of IoT devices is increasing at a fast pace, whether for private or business use. However,despite their growing popularity, their safe operation is often not guaranteed. To tackle the security challenges of modern IoT environments, the German Federal Ministry of Education and Research is funding the SIMPL project in order to support research in this area. The contribution of this paper is to introduce the main goal, objectives, and key features of SIMPL}, author = {Prantl, Thomas and Ben Yahya, Ala Eddine and Dmitrienko, Alexandra and Kounev, Samuel and Lipp, Fabian and Hock, David and Rathfelder, Christoph and Hofherr, Martin}, booktitle = {ITG Workshop on IT Security (ITSec)}, keywords = {t_workshop}, title = {SIMPL: Secure IoT Management Platform}, year = 2020 }
• 
  Predicting Performance Degradations of Black-box Microservice Applications M. Straesser; (2020).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @presentation{straesser2020talk, author = {Straesser, Martin}, keywords = {sys:relevantfor:se-group}, title = {Predicting Performance Degradations of Black-box Microservice Applications}, year = 2020 }
• 
  All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers. C. Hagen; C. Weinert; C. Sendner; A. Dmitrienko; T. Schneider; in Cryptology ePrint Archive, Report 2020/1119 (2020).
  • [ BibTeX ]
  • [ URL ]
  @article{cryptoeprint:2020:1119, author = {Hagen, Christoph and Weinert, Christian and Sendner, Christoph and Dmitrienko, Alexandra and Schneider, Thomas}, journal = {Cryptology ePrint Archive, Report 2020/1119}, keywords = {security}, title = {All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers}, year = 2020 }
• 
  Intra-individual physiological response of recreational runners to different training mesocycles: a randomized cross-over study. P. Düking; H.-C. Holmberg; P. Kunz; R. Leppich; B. Sperlich; in European Journal of Applied Physiology (2020). 1–9.
  • [ BibTeX ]
  @article{duking2020intra, author = {D{\"u}king, Peter and Holmberg, Hans-Christer and Kunz, Philipp and Leppich, Robert and Sperlich, Billy}, journal = {European Journal of Applied Physiology}, keywords = {t_journalmagazine}, pages = {1–9}, publisher = {Springer}, title = {Intra-individual physiological response of recreational runners to different training mesocycles: a randomized cross-over study}, year = 2020 }
• 
  An Overview of Design Patterns for Self-Adaptive Systems in the Context of the Internet of Things. C. Krupitzer; T. Temizer; T. Prantl; C. Raibulet; in IEEE Access (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The Internet of Things (IoT) requires the integration of all available, highly specialized, and heterogeneous devices, ranging from embedded sensor nodes to servers in the cloud. The self-adaptive research domain provides adaptive capabilities that can support the integration in IoT systems. However, developing such systems is a challenging, error-prone, and time-consuming task. In this context, design patterns propose already used and optimized solutions to specific problems in various contexts. Applying design patterns might help to reuse existing knowledge about similar development issues. However, so far, there is a lack of taxonomies on design patterns for self-adaptive systems. To tackle this issue, in this paper, we provide a taxonomy on design patterns for self-adaptive systems that can be transferred to support adaptivity in IoT systems. Besides describing the taxonomy and the design patterns, we discuss their applicability in an Industrial IoT case study.

  @article{ieeeaccess2020:sasdesignpatterns, abstract = {The Internet of Things (IoT) requires the integration of all available, highly specialized, and heterogeneous devices, ranging from embedded sensor nodes to servers in the cloud. The self-adaptive research domain provides adaptive capabilities that can support the integration in IoT systems. However, developing such systems is a challenging, error-prone, and time-consuming task. In this context, design patterns propose already used and optimized solutions to specific problems in various contexts. Applying design patterns might help to reuse existing knowledge about similar development issues. However, so far, there is a lack of taxonomies on design patterns for self-adaptive systems. To tackle this issue, in this paper, we provide a taxonomy on design patterns for self-adaptive systems that can be transferred to support adaptivity in IoT systems. Besides describing the taxonomy and the design patterns, we discuss their applicability in an Industrial IoT case study.}, author = {Krupitzer, Christian and Temizer, Timur and Prantl, Thomas and Raibulet, Claudia}, journal = {IEEE Access}, keywords = {self-adaptive-systems}, title = {An Overview of Design Patterns for Self-Adaptive Systems in the Context of the Internet of Things}, year = 2020 }
• 
  LegIoT: Ledgered Trust Management Platform for IoT. J. Neureither; A. Dmitrienko; D. Koisser; F. Brasser; A.-R. Sadeghi; in European Symposium on Research in Computer Security (ESORICS) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  We investigate and address the currently unsolved problem of trust establishment in large-scale Internet of Things (IoT) networks where heterogeneous devices and mutually mistrusting stakeholders are involved. We design, prototype and evaluate LegIoT, a novel, probabilistic trust management system that enables secure, dynamic and flexible(yet inexpensive) trust relationships in large IoT networks. The core component of LegIoT is a novel graph-based scheme that allows network devices (graph nodes) to re-use the already existing trust associations(graph edges) very efficiently; thus, significantly reducing the number of individually conducted trust assessments. Since no central trusted third party exists, LegIoT leverages Distributed Ledger Technology(DLT) to create and manage the trust relation graph in a decentralized manner.The trust assessment among devices can be instantiated by any appropriate assessment technique, for which we focus on remote attestation(integrity verification) in this paper. We prototyped LegIoT for Hyper-ledger Sawtooth and demonstrated through evaluation that the number of trust assessments in the network can be significantly reduced – e.g.,by a factor of 20 for a network of 400 nodes and factor 5 for 1000 nodes.

  @inproceedings{neureither2020legiot, abstract = {We investigate and address the currently unsolved problem of trust establishment in large-scale Internet of Things (IoT) networks where heterogeneous devices and mutually mistrusting stakeholders are involved. We design, prototype and evaluate LegIoT, a novel, probabilistic trust management system that enables secure, dynamic and flexible(yet inexpensive) trust relationships in large IoT networks. The core component of LegIoT is a novel graph-based scheme that allows network devices (graph nodes) to re-use the already existing trust associations(graph edges) very efficiently; thus, significantly reducing the number of individually conducted trust assessments. Since no central trusted third party exists, LegIoT leverages Distributed Ledger Technology(DLT) to create and manage the trust relation graph in a decentralized manner.The trust assessment among devices can be instantiated by any appropriate assessment technique, for which we focus on remote attestation(integrity verification) in this paper. We prototyped LegIoT for Hyper-ledger Sawtooth and demonstrated through evaluation that the number of trust assessments in the network can be significantly reduced – e.g.,by a factor of 20 for a network of 400 nodes and factor 5 for 1000 nodes.}, author = {Neureither, Jens and Dmitrienko, Alexandra and Koisser, David and Brasser, Ferdinand and Sadeghi, Ahmad-Reza}, booktitle = {European Symposium on Research in Computer Security (ESORICS)}, keywords = {sss-group}, title = {LegIoT: Ledgered Trust Management Platform for IoT}, year = 2020 }
• 
  Quantifying measurement quality and load distribution in Tor. A. Greubel; S. Pohl; S. Kounev; in Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.

  @article{greubel2020quantifying, abstract = {Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80\%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21\% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.}, author = {Greubel, Andre and Pohl, Steffen and Kounev, Samuel}, journal = {Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020)}, keywords = {tor}, title = {Quantifying measurement quality and load distribution in Tor}, year = 2020 }
• 
  Learning to Learn in Collective Adaptive Systems: Mining Design Pattern for Data-driven Reasoning. M. D’Angelo; S. Ghahremani; S. Gerasimou; J. Grohmann; I. Nunes; S. Tomforde; E. Pournaras; in 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) (2020). 121–126.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Engineering collective adaptive systems (CAS) with learning capabilities is a challenging task due to their multidimensional and complex design space. Data-driven approaches for CAS design could introduce new insights enabling system engineers to manage the CAS complexity more cost-effectively at the design-phase. This paper introduces a systematic approach to reason about design choices and patterns of learning-based CAS. Using data from a systematic literature review, reasoning is performed with a novel application of data-driven methodologies such as clustering, multiple correspondence analysis and decision trees. The reasoning based on past experience as well as supporting novel and innovative design choices are demonstrated.

  @conference{DaGhGeGrNuToPo2020-eCas-LearningInCAS, abstract = {Engineering collective adaptive systems (CAS) with learning capabilities is a challenging task due to their multidimensional and complex design space. Data-driven approaches for CAS design could introduce new insights enabling system engineers to manage the CAS complexity more cost-effectively at the design-phase. This paper introduces a systematic approach to reason about design choices and patterns of learning-based CAS. Using data from a systematic literature review, reasoning is performed with a novel application of data-driven methodologies such as clustering, multiple correspondence analysis and decision trees. The reasoning based on past experience as well as supporting novel and innovative design choices are demonstrated.}, author = {D'Angelo, Mirko and Ghahremani, Sona and Gerasimou, Simos and Grohmann, Johannes and Nunes, Ingrid and Tomforde, Sven and Pournaras, Evangelos}, booktitle = {2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)}, keywords = {t_workshop}, pages = {121–126}, publisher = {IEEE}, title = {Learning to Learn in Collective Adaptive Systems: Mining Design Pattern for Data-driven Reasoning}, year = 2020 }
• 
  Toward a Framework for Self-Learning Adaptation Planning through Optimization. V. Lesch; in Organic Computing: Doctoral Dissertation Colloquium 2020 (2020).
  • [ BibTeX ]
  @incollection{lesch2020toward, author = {Lesch, Veronika}, booktitle = {Organic Computing: Doctoral Dissertation Colloquium 2020}, keywords = {cyber-physical_systems}, title = {Toward a Framework for Self-Learning Adaptation Planning through Optimization}, year = 2020 }
• 
  The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study. C. Zinner; M. Matzka; R. Leppich; S. Kounev; H. Holmberg; B. Sperlich; in Frontiers in Sports and Active Living (2020). 2 127.
  • [ BibTeX ]
  @article{zinner2020impact, author = {Zinner, Christoph and Matzka, Manuel and Leppich, Robert and Kounev, Samuel and Holmberg, HC and Sperlich, Billy}, journal = {Frontiers in Sports and Active Living}, keywords = {covid19}, pages = 127, publisher = {Frontiers}, title = {The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study}, volume = 2, year = 2020 }
• 
  Methodological Principles for Reproducible Performance Evaluation in Cloud Computing. A. V. Papadopoulos; L. Versluis; A. Bauer; N. Herbst; J. von Kistowski; A. Ali-Eldin; C. L. Abad; J. N. Amaral; P. Tuma; A. Iosup; in Software Engineering 2020, Fachtagung des GI-Fachbereichs Softwaretechnik, 24.-28. Februar 2020, Innsbruck, Austria, M. Felderer, W. Hasselbring, R. Rabiser, R. Jung (eds.) (2020). (Vol. P-300) 93–94.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{DBLP:conf/se/PapadopoulosVBH20, author = {Papadopoulos, Alessandro Vittorio and Versluis, Laurens and Bauer, Andr{{\'e}} and Herbst, Nikolas and von Kistowski, J{{\'o}}akim and Ali{-}Eldin, Ahmed and Abad, Cristina L. and Amaral, Jos{{\'e}} Nelson and Tuma, Petr and Iosup, Alexandru}, booktitle = {Software Engineering 2020, Fachtagung des GI-Fachbereichs Softwaretechnik, 24.-28. Februar 2020, Innsbruck, Austria}, editor = {Felderer, Michael and Hasselbring, Wilhelm and Rabiser, Rick and Jung, Reiner}, keywords = {se2}, pages = {93–94}, publisher = {Gesellschaft f{{\"u}}r Informatik e.V.}, series = {{LNI}}, title = {{Methodological Principles for Reproducible Performance Evaluation in Cloud Computing}}, volume = {{P-300}}, year = 2020 }
• 
  Model-based Performance Predictions for SDN-based Networks: A Case Study. S. Herrnleben; P. Rygielski; J. Grohmann; S. Eismann; T. Hossfeld; S. Kounev; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.

  @inproceedings{HeRyGrEiHoKo-MMB2020-Model-based-SDN-Performance, abstract = {Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Rygielski, Piotr and Grohmann, Johannes and Eismann, Simon and Hossfeld, Tobias and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {SDN}, month = {03}, publisher = {Springer}, series = {MMB 2020}, title = {{Model-based Performance Predictions for SDN-based Networks: A Case Study}}, year = 2020 }
• 
  Incremental Calibration of Architectural Performance Models with Parametric Dependencies. M. Mazkatli; D. Monschein; J. Grohmann; A. Koziolek; in 2020 IEEE International Conference on Software Architecture (ICSA 2020) (2020). 23–34.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Architecture-based Performance Prediction (AbPP) allows evaluation of the performance of systems and to answer what-if questions without measurements for all alternatives. A difficulty when creating models is that Performance Model Parameters (PMPs, such as resource demands, loop iteration numbers and branch probabilities) depend on various influencing factors like input data, used hardware and the applied workload. To enable a broad range of what-if questions, Performance Models (PMs) need to have predictive power beyond what has been measured to calibrate the models. Thus, PMPs need to be parametrized over the influencing factors that may vary. Existing approaches allow for the estimation of parametrized PMPs by measuring the complete system. Thus, they are too costly to be applied frequently, up to after each code change. They do not keep also manual changes to the model when recalibrating. In this work, we present the Continuous Integration of Performance Models (CIPM), which incrementally extracts and calibrates the performance model, including parametric dependencies. CIPM responds to source code changes by updating the PM and adaptively instrumenting the changed parts. To allow AbPP, CIPM estimates the parametrized PMPs using the measurements (generated by performance tests or executing the system in production) and statistical analysis, e.g., regression analysis and decision trees. Additionally, our approach responds to production changes (e.g., load or deployment changes) and calibrates the usage and deployment parts of PMs accordingly. For the evaluation, we used two case studies. Evaluation results show that we were able to calibrate the PM incrementally and accurately.

  @inproceedings{MaMoGrKo2020-ICSA, abstract = {Architecture-based Performance Prediction (AbPP) allows evaluation of the performance of systems and to answer what-if questions without measurements for all alternatives. A difficulty when creating models is that Performance Model Parameters (PMPs, such as resource demands, loop iteration numbers and branch probabilities) depend on various influencing factors like input data, used hardware and the applied workload. To enable a broad range of what-if questions, Performance Models (PMs) need to have predictive power beyond what has been measured to calibrate the models. Thus, PMPs need to be parametrized over the influencing factors that may vary. Existing approaches allow for the estimation of parametrized PMPs by measuring the complete system. Thus, they are too costly to be applied frequently, up to after each code change. They do not keep also manual changes to the model when recalibrating. In this work, we present the Continuous Integration of Performance Models (CIPM), which incrementally extracts and calibrates the performance model, including parametric dependencies. CIPM responds to source code changes by updating the PM and adaptively instrumenting the changed parts. To allow AbPP, CIPM estimates the parametrized PMPs using the measurements (generated by performance tests or executing the system in production) and statistical analysis, e.g., regression analysis and decision trees. Additionally, our approach responds to production changes (e.g., load or deployment changes) and calibrates the usage and deployment parts of PMs accordingly. For the evaluation, we used two case studies. Evaluation results show that we were able to calibrate the PM incrementally and accurately.}, author = {Mazkatli, Manar and Monschein, David and Grohmann, Johannes and Koziolek, Anne}, booktitle = {2020 IEEE International Conference on Software Architecture (ICSA 2020)}, keywords = {performance}, month = {03}, pages = {23–34}, publisher = {IEEE}, title = {Incremental Calibration of Architectural Performance Models with Parametric Dependencies}, year = 2020 }
• 
  Predicting the Costs of Serverless Workflows. S. Eismann; J. Grohmann; E. van Eyk; N. Herbst; S. Kounev; in Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) (2020). 265–276.

  Acceptance Rate: 23.4% (15/64)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.

  @inproceedings{EiGrEyHeKo2020-ICPE-ServerlessWorkflows, abstract = {Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.}, address = {New York, NY, USA}, author = {Eismann, Simon and Grohmann, Johannes and van Eyk, Erwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {se2}, month = {04}, note = {{Acceptance Rate: 23.4% (15/64)}}, pages = {265–276}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {Predicting the Costs of Serverless Workflows}, year = 2020 }
• 
  Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario. T. Prantl; P. Ten; L. Iffländer; A. Dmitrenko; S. Kounev; C. Krupitzer; in 2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020).

  Acceptance Rate: 27%
  • [ Abstract ]
  • [ BibTeX ]
  New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.

  @inproceedings{PrTeIfDmKo2020-MASCOTS-GroupEncryption, abstract = {New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.}, author = {Prantl, Thomas and Ten, Peter and Iffländer, Lukas and Dmitrenko, Alexandra and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {Performance_Evaluation}, month = 11, note = {Acceptance Rate: 27\%}, series = {MASCOTS '20}, title = {Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario}, year = 2020 }
• 
  A Framework for Time Series Preprocessing and History-based Forecasting Method Recommendation. M. Züfle; S. Kounev; in Proceedings of the 15th Conference on Computer Science and Information Systems (FedCSIS): Data Mining Competition of the International Symposium on Advanced Artificial Intelligence in Applications (2020).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{ZuKo-AAIA2020-Framework, author = {Z{\"u}fle, Marwin and Kounev, Samuel}, booktitle = {Proceedings of the 15th Conference on Computer Science and Information Systems (FedCSIS): Data Mining Competition of the International Symposium on Advanced Artificial Intelligence in Applications}, keywords = {online_monitoring_and_forecasting}, month = {09}, title = {A Framework for Time Series Preprocessing and History-based Forecasting Method Recommendation}, year = 2020 }
• 
  Understanding UI attacks on Android. S. Jasper; (2020, December).
  • [ Abstract ]
  • [ BibTeX ]
  In this thesis, we closely examine an important type of attack against Android smart- phones that exploit weaknesses in the user interface. In particular, we study, analyze and implement the “Keystroke Inference #3” attack from the “Cloak and Dagger” paper by Fratantonio et al. [1, 2], which enables an attacker to steal sensitive input such as pass- words. The attack takes advantage of a vulnerability that was subsequently patched on all newer Android versions. Yet, it still affects a significant user base that utilizes older devices. In this paper, we present the end-to-end attack implementation of the “Keystroke Inference #3” concept and elaborate on in-depth details. In order to make the attack fea- sible certain technical challenges needed to be solved, therefore our developed approaches are presented as well. After the evaluation of the results, we show that the implementa- tion is applicable to a wide range of Android versions. We then present our novel defense technique OverlayShifter, which fully prevents the attack while being independent of operating system modifications. Moreover, characteristics that facilitate the detection of the attack are discussed.

  @mastersthesis{jasper2020understanding, abstract = {In this thesis, we closely examine an important type of attack against Android smart- phones that exploit weaknesses in the user interface. In particular, we study, analyze and implement the “Keystroke Inference #3” attack from the “Cloak and Dagger” paper by Fratantonio et al. [1, 2], which enables an attacker to steal sensitive input such as pass- words. The attack takes advantage of a vulnerability that was subsequently patched on all newer Android versions. Yet, it still affects a significant user base that utilizes older devices. In this paper, we present the end-to-end attack implementation of the “Keystroke Inference #3” concept and elaborate on in-depth details. In order to make the attack fea- sible certain technical challenges needed to be solved, therefore our developed approaches are presented as well. After the evaluation of the results, we show that the implementa- tion is applicable to a wide range of Android versions. We then present our novel defense technique OverlayShifter, which fully prevents the attack while being independent of operating system modifications. Moreover, characteristics that facilitate the detection of the attack are discussed.}, author = {Jasper, Stang}, keywords = {sssgroup}, month = 12, title = {Understanding UI attacks on Android}, type = {Bachelor Thesis}, year = 2020 }
• 
  Microservices: A Performance Tester’s Dream or Nightmare?. S. Eismann; C.-P. Bezemer; W. Shang; D. Okanovic; A. van Hoorn; in Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) (2020).

  Acceptance Rate: 23.4% (15/64), ACM Artifacts Evaluated — Functional
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In recent years, there has been a shift in software development towards microservice-based architectures, which consist of small services that focus on one particular functionality. Many companies are migrating their applications to such architectures to reap the benefits of microservices, such as increased flexibility, scalability and a smaller granularity of the offered functionality by a service. On the one hand, the benefits of microservices for functional testing are often praised, as the focus on one functionality and their smaller granularity allow for more targeted and more convenient testing. On the other hand, using microservices has their consequences (both positive and negative) on other types of testing, such as performance testing. Performance testing is traditionally done by establishing the baseline performance of a software version, which is then used to compare the performance testing results of later software versions. However, as we show in this paper, establishing such a baseline performance is challenging in microservice applications. In this paper, we discuss the benefits and challenges of microservices from a performance testers point of view. Through a series of experiments on the TeaStore application, we demonstrate how microservices affect the performance testing process, and we demonstrate that it is not straightforward to achieve reliable performance testing results for a microservice application.

  @inproceedings{EiBeShOkHo2020-ICPE-ServerlessWorkflows, abstract = {In recent years, there has been a shift in software development towards microservice-based architectures, which consist of small services that focus on one particular functionality. Many companies are migrating their applications to such architectures to reap the benefits of microservices, such as increased flexibility, scalability and a smaller granularity of the offered functionality by a service. On the one hand, the benefits of microservices for functional testing are often praised, as the focus on one functionality and their smaller granularity allow for more targeted and more convenient testing. On the other hand, using microservices has their consequences (both positive and negative) on other types of testing, such as performance testing. Performance testing is traditionally done by establishing the baseline performance of a software version, which is then used to compare the performance testing results of later software versions. However, as we show in this paper, establishing such a baseline performance is challenging in microservice applications. In this paper, we discuss the benefits and challenges of microservices from a performance testers point of view. Through a series of experiments on the TeaStore application, we demonstrate how microservices affect the performance testing process, and we demonstrate that it is not straightforward to achieve reliable performance testing results for a microservice application.}, author = {Eismann, Simon and Bezemer, Cor-Paul and Shang, Weiyi and Okanovic, Dusan and van Hoorn, Andre}, booktitle = {Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {se2}, month = {04}, note = {{Acceptance Rate: 23.4% (15/64), ACM Artifacts Evaluated — Functional}}, series = {ICPE'20}, title = {{Microservices: A Performance Tester's Dream or Nightmare?}}, year = 2020 }
• 
  Contact Tracing by Giant Data Collectors: Opening Pandora’s Box of Threats to Privacy, Sovereignty and National Security A. Boutet; C. Castelluccia; M. Cunche; A. Dmitrienko; V. Iovino; M. Miettinen; T. D. Nguyen; V. Roca; A.-R. Sadeghi; S. Vaudenay; I. Visconti; M. Vuagnoux; (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Many countries have introduced digital contact tracing apps to fight the COVID-19 pandemic. Such apps help to identify contacts between potentially infectious persons automatically and thus bear the promise of reducing the burden on manual contact tracers and increase tracing accuracy in situations in which people have difficulties identifying with whom they have been in contact. A number of different proposals for digital contact tracing systems have been made or deployed, ranging from heavily centralized to completely decentralized approaches, each with its own advantages and disadvantages in terms of tracing effectiveness and impact on user privacy. During the phase of highly dynamic evolution of these approaches, surprisingly, Google and Apple established an unprecedented friendship and agreed on a very special scheme for contact tracing, realizing this in the form of an API called GAEN that they quickly integrated into their mobile operating systems. A multitude of nationally rolled out tracing apps are now based on the GAEN approach. In this paper, we revisit such apps and the GAEN API on which they are built. In particular, we point out a number of very problematic aspects and threats that the GAEN approach creates through its security and privacy weaknesses but also through the threats that it poses on technological sovereignty and the public health system.

  @techreport{boutet2020contact, abstract = {Many countries have introduced digital contact tracing apps to fight the COVID-19 pandemic. Such apps help to identify contacts between potentially infectious persons automatically and thus bear the promise of reducing the burden on manual contact tracers and increase tracing accuracy in situations in which people have difficulties identifying with whom they have been in contact. A number of different proposals for digital contact tracing systems have been made or deployed, ranging from heavily centralized to completely decentralized approaches, each with its own advantages and disadvantages in terms of tracing effectiveness and impact on user privacy. During the phase of highly dynamic evolution of these approaches, surprisingly, Google and Apple established an unprecedented friendship and agreed on a very special scheme for contact tracing, realizing this in the form of an API called GAEN that they quickly integrated into their mobile operating systems. A multitude of nationally rolled out tracing apps are now based on the GAEN approach. In this paper, we revisit such apps and the GAEN API on which they are built. In particular, we point out a number of very problematic aspects and threats that the GAEN approach creates through its security and privacy weaknesses but also through the threats that it poses on technological sovereignty and the public health system.}, author = {Boutet, Antoine and Castelluccia, Claude and Cunche, Mathieu and Dmitrienko, Alexandra and Iovino, Vincenzo and Miettinen, Markus and Nguyen, Thien Duc and Roca, Vincent and Sadeghi, Ahmad-Reza and Vaudenay, Serge and Visconti, Ivan and Vuagnoux, Martin}, institution = {{EPFL, Switzerland ; Inria, France ; JMU W{\"u}rzburg, Germany ; University of Salerno, Italy ; base23, Geneva, Switzerland ; Technical University of Darmstadt, Germany}}, keywords = {collector}, month = 12, title = {{Contact Tracing by Giant Data Collectors: Opening Pandora's Box of Threats to Privacy, Sovereignty and National Security}}, type = {University works}, year = 2020 }
• 
  Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data. F. Spiess; J. Friesslich; K. Tobias; S. Kounev; N. Strobel; in International Journal of Mechanical Engineering and Robotics Research (IJMERR), (P. R. (Chunhui) Yang, ed.) (2020). 9(12) 1532–1540.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.

  @article{yang2020survey, abstract = {This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.}, author = {Spiess, Florian and Friesslich, Jonas and Tobias, Kaupp and Kounev, Samuel and Strobel, Norbert}, editor = {Yang, Prof Richard (Chunhui)}, journal = {International Journal of Mechanical Engineering and Robotics Research (IJMERR)}, keywords = {rgb}, month = 12, number = 12, pages = {1532-1540}, title = {Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data}, volume = 9, year = 2020 }
• 
  Aggregatable Remote Attestation for IoT. V. Alistarov; Thesis; University of Würzburg. (2020, December).
  • [ Abstract ]
  • [ BibTeX ]
  With the recent spike of Internet of Things (IoT) and "smart" devices, there has also been an increase in the amount of attacks on IoT networks. Wide-reaching attacks such as the one from the Mirai botnet in 2016 show how crucial it is to know that a device can be trusted before initiating communication. Remote Attestation (RA) is a proven method for asserting that a device is in a benign state. It is a challenge-response process between two parties, where the first checks the trustworthiness of the second. However, it is characterizable with low scalability a critical issue in the IoT sector. In our work, we model a new RA protocol, called Aggregatable Remote Attestation, which would allow a device to process multiple RA challenges simultaneously. We base it on the already existing SIMPLE architecture and implement it as a Proof-of-Concept (PoC) by modifying the code of the Security MicroVisor the core component of SIMPLE. We evaluate our work in terms of security and performance and show that it greatly outperforms the underlying SIMPLE. We discuss the relevance of our design in relation to the IoT sphere and denote a small set of potential topics for future work and research.

  @mastersthesis{alistarov2020aggregatable, abstract = {With the recent spike of Internet of Things (IoT) and "smart" devices, there has also been an increase in the amount of attacks on IoT networks. Wide-reaching attacks such as the one from the Mirai botnet in 2016 show how crucial it is to know that a device can be trusted before initiating communication. Remote Attestation (RA) is a proven method for asserting that a device is in a benign state. It is a challenge-response process between two parties, where the first checks the trustworthiness of the second. However, it is characterizable with low scalability a critical issue in the IoT sector. In our work, we model a new RA protocol, called Aggregatable Remote Attestation, which would allow a device to process multiple RA challenges simultaneously. We base it on the already existing SIMPLE architecture and implement it as a Proof-of-Concept (PoC) by modifying the code of the Security MicroVisor the core component of SIMPLE. We evaluate our work in terms of security and performance and show that it greatly outperforms the underlying SIMPLE. We discuss the relevance of our design in relation to the IoT sphere and denote a small set of potential topics for future work and research.}, author = {Alistarov, Vasil}, keywords = {security}, month = 12, school = {University of W{\"u}rzburg}, title = {Aggregatable Remote Attestation for IoT}, type = {Bachelor Thesis}, year = 2020 }
• 
  Performance, Power, and Energy-Efficiency Impact Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite. N. Schmitt; J. Bucek; J. Beckett; A. Cragin; K.-D. Lange; S. Kounev; in 2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC) (2020). 292–301.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The growth of cloud services leads to more and more data centers that are increasingly larger and consume considerable amounts of power. To increase energy efficiency, both the actual server equipment and the software must become more energy efficient. Software has a major impact on hardware utilization levels, and subsequently, the energy efficiency. While energy efficiency is often seen as identical to performance, we argue that this may not be necessarily the case. A sizable amount of energy could be saved, increasing energy efficiency by leveraging compiler optimizations but at the same time impacting performance and power consumption over time. We analyze the SPEC CPU 2017 benchmark suite with 43 benchmarks from different domains, including integer and floating-point heavy computations on a state-of-the-art server system for cloud applications. Our results show that power consumption displays more stable behavior if less compiler optimizations are used and also confirmed that performance and energy efficiency are different optimizations goals. Additionally, compiler optimizations possibly could be used to enable power capping on a software level and care must be taken when selecting such optimizations.

  @inproceedings{9302787, abstract = {The growth of cloud services leads to more and more data centers that are increasingly larger and consume considerable amounts of power. To increase energy efficiency, both the actual server equipment and the software must become more energy efficient. Software has a major impact on hardware utilization levels, and subsequently, the energy efficiency. While energy efficiency is often seen as identical to performance, we argue that this may not be necessarily the case. A sizable amount of energy could be saved, increasing energy efficiency by leveraging compiler optimizations but at the same time impacting performance and power consumption over time. We analyze the SPEC CPU 2017 benchmark suite with 43 benchmarks from different domains, including integer and floating-point heavy computations on a state-of-the-art server system for cloud applications. Our results show that power consumption displays more stable behavior if less compiler optimizations are used and also confirmed that performance and energy efficiency are different optimizations goals. Additionally, compiler optimizations possibly could be used to enable power capping on a software level and care must be taken when selecting such optimizations.}, author = {Schmitt, Norbert and Bucek, James and Beckett, John and Cragin, Aaron and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {2020 IEEE/ACM 13th International Conference on Utility and Cloud Computing (UCC)}, keywords = {power}, month = 12, pages = {292-301}, title = {Performance, Power, and Energy-Efficiency Impact Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite}, year = 2020 }
• 
  Testbed for Security Testing of Smart Contracts. L. Denk; Thesis; University of Würzburg. (2020, November).
  • [ Abstract ]
  • [ BibTeX ]
  Ethereum, one of the most popular decentralized blockchain-based platforms [1], manages cryptocurrency worth more than 40 billion US Dollars [2]. A lot of the money is controlled by autonomous programs, so called smart contracts. They are executed on the platform and everyone can call their functions; vulnerabilities are therefore easily exploited. The immutable nature of the blockchain sharpens the problem even further by prohibiting belated bug fixes. In fact, there were several cases where attackers were able to steal cryptocurrency worth several millions of US Dollar [3]. This emphasises the need of soundly testing a smart contract before deployment. Unfortunately, existing security analysing tools each cover a different subsets of vulnerabilities [4]. Moreover, even when they test for the same security issues, they often find different results. A solid test run should therefore contain several tools, which makes the whole procedure very laborious. For these reasons, we provide a testbed which runs on a virtual machine (VM) with several smart contract security analysing tools integrated. It does not only offer a command line- but also an intuitive web interface. Both interfaces allow the user to analyse his contract with all the underlying tools in a single step. Additionally, our testbed provides a detailed report for each of the tools’ test runs as well as an overview of the overall security issues the different tools found. Our evaluation shows that our testbed identifies significantly more potential security issues than each of the compared smart contract analysing tools individually does. Indeed, we found 92% of our analysed contracts as potentially being vulnerable, while even the most sensitive tool only flagged 76% of its successfully analysed contracts. Furthermore, we observed that occasionally, two tools testing for the same vulnerability, contradict themselves in more than 81% of the contracts successfully analysed by both tools

  @mastersthesis{denk2020testbed, abstract = {Ethereum, one of the most popular decentralized blockchain-based platforms [1], manages cryptocurrency worth more than 40 billion US Dollars [2]. A lot of the money is controlled by autonomous programs, so called smart contracts. They are executed on the platform and everyone can call their functions; vulnerabilities are therefore easily exploited. The immutable nature of the blockchain sharpens the problem even further by prohibiting belated bug fixes. In fact, there were several cases where attackers were able to steal cryptocurrency worth several millions of US Dollar [3]. This emphasises the need of soundly testing a smart contract before deployment. Unfortunately, existing security analysing tools each cover a different subsets of vulnerabilities [4]. Moreover, even when they test for the same security issues, they often find different results. A solid test run should therefore contain several tools, which makes the whole procedure very laborious. For these reasons, we provide a testbed which runs on a virtual machine (VM) with several smart contract security analysing tools integrated. It does not only offer a command line- but also an intuitive web interface. Both interfaces allow the user to analyse his contract with all the underlying tools in a single step. Additionally, our testbed provides a detailed report for each of the tools’ test runs as well as an overview of the overall security issues the different tools found. Our evaluation shows that our testbed identifies significantly more potential security issues than each of the compared smart contract analysing tools individually does. Indeed, we found 92% of our analysed contracts as potentially being vulnerable, while even the most sensitive tool only flagged 76% of its successfully analysed contracts. Furthermore, we observed that occasionally, two tools testing for the same vulnerability, contradict themselves in more than 81% of the contracts successfully analysed by both tools}, author = {Denk, Lukas}, keywords = {secure}, month = 11, school = {University of W{\"u}rzburg}, title = {Testbed for Security Testing of Smart Contracts}, type = {Bachelor Thesis}, year = 2020 }
• 
  Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS. J. Grohmann; D. Seybold; S. Eismann; M. Leznik; S. Kounev; J. Domaschka; in 2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020). 1–8.

  Acceptance Rate: 27%
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.

  @inproceedings{GrSeEiLeKoDo2020-MASCOTS-DBMSPerformance, abstract = {Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.}, author = {Grohmann, Johannes and Seybold, Daniel and Eismann, Simon and Leznik, Mark and Kounev, Samuel and Domaschka, Jörg}, booktitle = {2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {automated_model_learning}, month = 11, note = {Acceptance Rate: 27%}, pages = {1–8}, publisher = {IEEE}, series = {MASCOTS '20}, title = {Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS}, year = 2020 }
• 
  Detection of Software Vulnerabilities in Smart Contracts using Deep Learning. O. Lutz; Thesis; University of Würzburg. (2020, October).
  • [ Abstract ]
  • [ BibTeX ]
  Der Missbrauch von Programmierfehlern in Kryptowährungsplattformen kann zu großen wirtschaftlichen Schäden fuhren. Deshalb wird in dieser Arbeit analysiert, wie Deep-Learning genutzt werden kann, um Schwachstellen in Ethereum Smart Contracts aufzudecken. Als Ergebnis ist ein Framework entwickelt worden, das sich um die Datengewinnung, Vorklassifizierung und Modellbildung kummert. Mit diesem Framework können Deep-Learning-Modelle entwickelt, trainiert und evaluiert werden, um einen KI-Scanner zu erstellen. Selbst bei einfachen neuronalen Netz-Architekturen erreicht der KI-Scanner, der acht verschiedene Arten von Schwachstellen erkennen kann, eine Genauigkeit von 98%. Dieser KI-Scanner kann zur weiteren Verwendung eingesetzt werden. Er kann jedoch auch um Schwachstellentypen erweitert werden, um ein noch mächtigeres Tool zu bilden.

  @mastersthesis{lutz2020detection, abstract = {Der Missbrauch von Programmierfehlern in Kryptowährungsplattformen kann zu großen wirtschaftlichen Schäden fuhren. Deshalb wird in dieser Arbeit analysiert, wie Deep-Learning genutzt werden kann, um Schwachstellen in Ethereum Smart Contracts aufzudecken. Als Ergebnis ist ein Framework entwickelt worden, das sich um die Datengewinnung, Vorklassifizierung und Modellbildung kummert. Mit diesem Framework können Deep-Learning-Modelle entwickelt, trainiert und evaluiert werden, um einen KI-Scanner zu erstellen. Selbst bei einfachen neuronalen Netz-Architekturen erreicht der KI-Scanner, der acht verschiedene Arten von Schwachstellen erkennen kann, eine Genauigkeit von 98%. Dieser KI-Scanner kann zur weiteren Verwendung eingesetzt werden. Er kann jedoch auch um Schwachstellentypen erweitert werden, um ein noch mächtigeres Tool zu bilden.}, author = {Lutz, Oliver}, keywords = {smart-contracts}, month = 10, school = {University of W{\"u}rzburg}, title = {Detection of Software Vulnerabilities in Smart Contracts using Deep Learning}, type = {Master Thesis}, year = 2020 }
• 
  Optimizing Parametric Dependencies for Incremental Performance Model Extraction. S. Voneva; M. Mazkatli; J. Grohmann; A. Koziolek; in Companion of the 14th European Conference Software Architecture (ECSA 2020), H. Muccini, P. Avgeriou, B. Buhnova, J. Cámara, M. Caporuscio, M. Franzago, A. Koziolek, P. Scandurra, C. Trubiani, D. Weyns, U. Zdun (eds.) (2020). (Vol. 1269) 228–240.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Model-based performance prediction in agile software development promises to evaluate design alternatives and to reduce the cost of performance tests. To minimize the differences between a real software and its performance model, parametric dependencies are introduced. They express how the performance model parameters (such as loop iteration count, branch transition probabilities, resource demands, and external service call arguments) depend on impacting factors like the input data. The approaches that perform model-based performance prediction in agile software development have two major shortcomings: they are either costly because they do not update the performance models automatically after each commit, or do not consider more complex parametric dependencies than linear. This work extends an approach for continuous integration of performance model during agile development. Our extension aims to optimize the learning of parametric dependencies with a genetic programming algorithm to be able to detect non-linear dependencies. The case study results show that using genetic programming enables detecting more complex dependencies and improves the accuracy of the updated performance model.

  @inproceedings{VoMaGrKo-ECSA-Dependencies, abstract = {Model-based performance prediction in agile software development promises to evaluate design alternatives and to reduce the cost of performance tests. To minimize the differences between a real software and its performance model, parametric dependencies are introduced. They express how the performance model parameters (such as loop iteration count, branch transition probabilities, resource demands, and external service call arguments) depend on impacting factors like the input data. The approaches that perform model-based performance prediction in agile software development have two major shortcomings: they are either costly because they do not update the performance models automatically after each commit, or do not consider more complex parametric dependencies than linear. This work extends an approach for continuous integration of performance model during agile development. Our extension aims to optimize the learning of parametric dependencies with a genetic programming algorithm to be able to detect non-linear dependencies. The case study results show that using genetic programming enables detecting more complex dependencies and improves the accuracy of the updated performance model.}, address = {Cham}, author = {Voneva, Sonya and Mazkatli, Manar and Grohmann, Johannes and Koziolek, Anne}, booktitle = {Companion of the 14th European Conference Software Architecture ({ECSA} 2020)}, editor = {Muccini, Henry and Avgeriou, Paris and Buhnova, Barbora and C{\'{a}}mara, Javier and Caporuscio, Mauro and Franzago, Mirco and Koziolek, Anne and Scandurra, Patrizia and Trubiani, Catia and Weyns, Danny and Zdun, Uwe}, keywords = {t_workshop}, month = {09}, pages = {228–240}, publisher = {Springer}, series = {Communications in Computer and Information Science}, title = {Optimizing Parametric Dependencies for Incremental Performance Model Extraction}, volume = 1269, year = 2020 }
• 
  Mind the GAP: Security & Privacy Risks of Contact Tracing Apps. L. Baumgärtner; A. Dmitrienko; B. Freisleben; J. Höchst; M. Mezini; M. Miettinen; T. D. Nguyen; A. Penning; F. Roos; A.-R. Sadeghi; M. Schwarz; C. Uhl; (2020).

  Accepted to TrustCom 2020, Security Track
  • [ BibTeX ]
  @inproceedings{baumgartner2020security, author = {Baumgärtner, Lars and Dmitrienko, Alexandra and Freisleben, Bernd and Höchst, Jonas and Mezini, Mira and Miettinen, Markus and Nguyen, Thien Duc and Penning, Alvar and Roos, Filipp and Sadeghi, Ahmad-Reza and Schwarz, Michael and Uhl, Christian}, keywords = {sss-group}, month = {09}, note = {Accepted to TrustCom 2020, Security Track}, title = {Mind the GAP: Security & Privacy Risks of Contact Tracing Apps}, year = 2020 }
• 
  An IoT Network Emulator for Analyzing the Influence of Varying Network Quality. S. Herrnleben; R. Ailabouni; J. Grohmann; T. Prantl; C. Krupitzer; S. Kounev; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.

  @inproceedings{HeAiGrPrKrKo2020-SIMUtools-IoT-Network-Emulator, abstract = {IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.}, author = {Herrnleben, Stefan and Ailabouni, Rudy and Grohmann, Johannes and Prantl, Thomas and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, keywords = {networking}, month = {08}, series = {SIMUtools 2020}, title = {An IoT Network Emulator for Analyzing the Influence of Varying Network Quality}, year = 2020 }
• 
  Time Series Forecasting for Self-Aware Systems. A. Bauer; M. Züfle; N. Herbst; A. Zehe; A. Hotho; S. Kounev; in Proceedings of the IEEE (2020). 108(7) 1068–1093.
  • [ BibTeX ]
  • [ DOI ]
  @article{BaZuHeZeHoKo-PIEEE-Forecasting, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Zehe, Albin and Hotho, Andreas and Kounev, Samuel}, journal = {Proceedings of the IEEE}, keywords = {t_journalmagazine}, month = {07}, number = 7, pages = {1068 - 1093}, title = {{Time Series Forecasting for Self-Aware Systems}}, volume = 108, year = 2020 }
• 
  An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series. A. Bauer; M. Züfle; J. Grohmann; N. Schmitt; N. Herbst; S. Kounev; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2020). 48–55.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.

  @inproceedings{BaZuGrScHeKo-ICPE20-Seasonal-Forecast, abstract = {Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.}, address = {New York, NY, USA}, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Grohmann, Johannes and Schmitt, Norbert and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {PRISMA}, month = {04}, pages = {48–55}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {{An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series}}, year = 2020 }
• 
  Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite. N. Schmitt; J. Bucek; K.-D. Lange; S. Kounev; in Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020) (2020).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{ScLaBuKo-ICPE20-SPEC-CPU2017, address = {New York, NY, USA}, author = {Schmitt, Norbert and Bucek, James and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020)}, keywords = {Power}, month = {04}, publisher = {ACM}, title = {{Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite}}, year = 2020 }
• 
  SPEC Research Group Newsletter, vol. 3 no. 1. S. Kounev; A. van Hoorn; A. Bauer; N. Herbst; (2020, April).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{kounev2020research, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{\'e} and Herbst, Nikolas}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 3 no. 1}}, year = 2020 }
• 
  Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field. A. Bauer; M. Züfle; N. Herbst; S. Kounev; V. Curtef; in Proceedings of the 36th International Conference on Data Engineering (ICDE) (2020).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{BaZuHeKoCu-ICDE-Telescope, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 36th International Conference on Data Engineering (ICDE)}, keywords = {Prediction}, month = {04}, title = {Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field}, year = 2020 }
• 
  Strategies for the Security Assessment of IoT Devices by Certification Authorities. M. A. Finke; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2020, May).
  • [ Abstract ]
  • [ BibTeX ]
  The Internet of Things (IoT) is a global infrastructure that interconnects physical and virtual things based on evolving information and communication technologies. Along with the rise of the IoT, multiple unprecedented forms of security issues and attacks have emerged. While their limited resources disallow the utilization of computing-intensive security measures, the services of IoT devices represent an attractive target for vulnerability exploitation. Certification Authorities (CAs) have responded to the trend and offer security assessments and certification for IoT devices. To ensure qualitative and fair certification, security testing requires well-defined strategies. This bachelor thesis analyzes the existing IoT security assessment models and proposes a novel, CA oriented Testing Guide Model (TGM) for standardized and reproducible assessments. The TGM describes a complete security assessment approach for application in IoT device security certification procedures of CAs. As part of the bachelor thesis, the TGM is specified, implemented, and its applicability evaluated with the help of existing IoT devices. CAs account for the security of certified products. The certification of devices that (subsequently) prove to be insecure can lead to a consumer's loss of trust in the CA. Therefore, the decision for issuing certificates must be comprehensible and made carefully. For the risk estimation of security issues and vulnerabilities, it is common to make use of numerical scoring systems. Well-defined scoring systems allow precise and reproducible estimations. In addition to the introduction of the TGM, this bachelor thesis analyzes the scoring systems established in the field of IT security in regard to their applicability for the scoring of IoT related vulnerabilities, complete IoT devices, and for the specific requirements of CAs. This thesis uncovers incompatibilities and insufficiencies in the existing systems that disallow application for the scoring of IoT devices and for application by CAs. To counter these issues, the thesis introduces a novel Security Scoring System (SSS) for rating IoT devices based on the risks of their vulnerabilities, services, and operational contexts. The SSS is integrated into the TGM, implemented in software, and evaluated with a comparison to the existing scoring systems. To summarize, the contributions of this bachelor thesis are two novel models for the security assessment of devices of the IoT: the TGM and the SSS. The models are intended to find application for CAs and both standardize and improve the security testing and assessment procedures.

  @mastersthesis{BA-Finke-2020, abstract = {The Internet of Things (IoT) is a global infrastructure that interconnects physical and virtual things based on evolving information and communication technologies. Along with the rise of the IoT, multiple unprecedented forms of security issues and attacks have emerged. While their limited resources disallow the utilization of computing-intensive security measures, the services of IoT devices represent an attractive target for vulnerability exploitation. Certification Authorities (CAs) have responded to the trend and offer security assessments and certification for IoT devices. To ensure qualitative and fair certification, security testing requires well-defined strategies. This bachelor thesis analyzes the existing IoT security assessment models and proposes a novel, CA oriented Testing Guide Model (TGM) for standardized and reproducible assessments. The TGM describes a complete security assessment approach for application in IoT device security certification procedures of CAs. As part of the bachelor thesis, the TGM is specified, implemented, and its applicability evaluated with the help of existing IoT devices. CAs account for the security of certified products. The certification of devices that (subsequently) prove to be insecure can lead to a consumer's loss of trust in the CA. Therefore, the decision for issuing certificates must be comprehensible and made carefully. For the risk estimation of security issues and vulnerabilities, it is common to make use of numerical scoring systems. Well-defined scoring systems allow precise and reproducible estimations. In addition to the introduction of the TGM, this bachelor thesis analyzes the scoring systems established in the field of IT security in regard to their applicability for the scoring of IoT related vulnerabilities, complete IoT devices, and for the specific requirements of CAs. This thesis uncovers incompatibilities and insufficiencies in the existing systems that disallow application for the scoring of IoT devices and for application by CAs. To counter these issues, the thesis introduces a novel Security Scoring System (SSS) for rating IoT devices based on the risks of their vulnerabilities, services, and operational contexts. The SSS is integrated into the TGM, implemented in software, and evaluated with a comparison to the existing scoring systems. To summarize, the contributions of this bachelor thesis are two novel models for the security assessment of devices of the IoT: the TGM and the SSS. The models are intended to find application for CAs and both standardize and improve the security testing and assessment procedures.}, address = {Am Hubland, Informatikgebäude, 97074 Würzburg, Germany}, author = {Finke, Moritz Anton}, editor = {Dmitrienko, Alexandra}, keywords = {sss-group}, month = {05}, school = {University of Würzburg}, title = {Strategies for the Security Assessment of IoT Devices by Certification Authorities}, type = {Bachelor Thesis}, year = 2020 }
• 
  Strategies for the Security Assessment of IoT Devices by Certification Authorities. M. A. Finke; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2020, May).
  • [ Abstract ]
  • [ BibTeX ]
  The Internet of Things (IoT) is a global infrastructure that interconnects physical and virtual things based on evolving information and communication technologies. Along with the rise of the IoT, multiple unprecedented forms of security issues and attacks have emerged. While their limited resources disallow the utilization of computing-intensive security measures, the services of IoT devices represent an attractive target for vulnerability exploitation. Certification Authorities (CAs) have responded to the trend and offer security assessments and certification for IoT devices. To ensure qualitative and fair certification, security testing requires well-defined strategies. This bachelor thesis analyzes the existing IoT security assessment models and proposes a novel, CA oriented Testing Guide Model (TGM) for standardized and reproducible assessments. The TGM describes a complete security assessment approach for application in IoT device security certification procedures of CAs. As part of the bachelor thesis, the TGM is specified, implemented, and its applicability evaluated with the help of existing IoT devices. CAs account for the security of certified products. The certification of devices that (subsequently) prove to be insecure can lead to a consumer's loss of trust in the CA. Therefore, the decision for issuing certificates must be comprehensible and made carefully. For the risk estimation of security issues and vulnerabilities, it is common to make use of numerical scoring systems. Well-defined scoring systems allow precise and reproducible estimations. In addition to the introduction of the TGM, this bachelor thesis analyzes the scoring systems established in the field of IT security in regard to their applicability for the scoring of IoT related vulnerabilities, complete IoT devices, and for the specific requirements of CAs. This thesis uncovers incompatibilities and insufficiencies in the existing systems that disallow application for the scoring of IoT devices and for application by CAs. To counter these issues, the thesis introduces a novel Security Scoring System (SSS) for rating IoT devices based on the risks of their vulnerabilities, services, and operational contexts. The SSS is integrated into the TGM, implemented in software, and evaluated with a comparison to the existing scoring systems. To summarize, the contributions of this bachelor thesis are two novel models for the security assessment of devices of the IoT: the TGM and the SSS. The models are intended to find application for CAs and both standardize and improve the security testing and assessment procedures.

  @mastersthesis{finke2020strategies, abstract = {The Internet of Things (IoT) is a global infrastructure that interconnects physical and virtual things based on evolving information and communication technologies. Along with the rise of the IoT, multiple unprecedented forms of security issues and attacks have emerged. While their limited resources disallow the utilization of computing-intensive security measures, the services of IoT devices represent an attractive target for vulnerability exploitation. Certification Authorities (CAs) have responded to the trend and offer security assessments and certification for IoT devices. To ensure qualitative and fair certification, security testing requires well-defined strategies. This bachelor thesis analyzes the existing IoT security assessment models and proposes a novel, CA oriented Testing Guide Model (TGM) for standardized and reproducible assessments. The TGM describes a complete security assessment approach for application in IoT device security certification procedures of CAs. As part of the bachelor thesis, the TGM is specified, implemented, and its applicability evaluated with the help of existing IoT devices. CAs account for the security of certified products. The certification of devices that (subsequently) prove to be insecure can lead to a consumer's loss of trust in the CA. Therefore, the decision for issuing certificates must be comprehensible and made carefully. For the risk estimation of security issues and vulnerabilities, it is common to make use of numerical scoring systems. Well-defined scoring systems allow precise and reproducible estimations. In addition to the introduction of the TGM, this bachelor thesis analyzes the scoring systems established in the field of IT security in regard to their applicability for the scoring of IoT related vulnerabilities, complete IoT devices, and for the specific requirements of CAs. This thesis uncovers incompatibilities and insufficiencies in the existing systems that disallow application for the scoring of IoT devices and for application by CAs. To counter these issues, the thesis introduces a novel Security Scoring System (SSS) for rating IoT devices based on the risks of their vulnerabilities, services, and operational contexts. The SSS is integrated into the TGM, implemented in software, and evaluated with a comparison to the existing scoring systems. To summarize, the contributions of this bachelor thesis are two novel models for the security assessment of devices of the IoT: the TGM and the SSS. The models are intended to find application for CAs and both standardize and improve the security testing and assessment procedures.}, address = {Am Hubland, Informatikgebäude, 97074 Würzburg, Germany}, author = {Finke, Moritz Anton}, keywords = {IoT}, month = {05}, school = {University of Würzburg}, title = {Strategies for the Security Assessment of IoT Devices by Certification Authorities}, type = {Bachelor Thesis}, year = 2020 }
• 
  Mind the GAP: Security & Privacy Risks of Contact Tracing Apps. L. Baumgärtner; A. Dmitrienko; B. Freisleben; A. Gruler; J. Höchst; J. Kühlberg; M. Mezini; M. Miettinen; A. Muhamedagic; T. D. Nguyen; A. Penning; D. F. Pustelnik; F. Roos; A.-R. Sadeghi; M. Schwarz; C. Uhl; in ArXiv (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Contact tracing apps running on mobile devices promise to reduce the manual effort required for identifying infection chains and to increase the tracing accuracy in the presence of COVID-19. Several contract tracing apps have been proposed or deployed in practice. Also Google and Apple have announced their joint effort of providing an API for exposure notification in order to implement decentralized contract tracing apps using Bluetooth Low Energy, the so-called "Google/Apple Proposal", which we abbreviate by "GAP". Some countries have already decided or are planning to base their contact tracing apps on GAP. Several researchers have pointed out potential privacy and security risks related to most of the contact tracing approaches proposed until now, including those that claim privacy protection and are based on GAP. This report makes a first attempt towards providing empirical evidence in real-world scenarios for two such risks discussed in the literature: one concerning privacy, and the other one concerning security. In particular, we focus on a practical analysis of GAP, given that it is the foundation of several tracing apps. We demonstrate that in real-world scenarios the current GAP design is vulnerable to (i) profiling and possibly de-anonymizing infected persons, and (ii) relay-based wormhole attacks that principally can generate fake contacts with the potential of significantly affecting the accuracy of an app-based contact tracing system. For both types of attack, we have built tools that can be easily used on mobile phones or Raspberry Pis (e.g., Bluetooth sniffers). We hope that our findings provide valuable input in the process of testing and certifying contact tracing apps, e.g., as planned for the German Corona-Warn-App, ultimately guiding improvements for secure and privacy-preserving design and implementation of digital contact tracing systems.

  @article{mindthegap, abstract = {Contact tracing apps running on mobile devices promise to reduce the manual effort required for identifying infection chains and to increase the tracing accuracy in the presence of COVID-19. Several contract tracing apps have been proposed or deployed in practice. Also Google and Apple have announced their joint effort of providing an API for exposure notification in order to implement decentralized contract tracing apps using Bluetooth Low Energy, the so-called "Google/Apple Proposal", which we abbreviate by "GAP". Some countries have already decided or are planning to base their contact tracing apps on GAP. Several researchers have pointed out potential privacy and security risks related to most of the contact tracing approaches proposed until now, including those that claim privacy protection and are based on GAP. This report makes a first attempt towards providing empirical evidence in real-world scenarios for two such risks discussed in the literature: one concerning privacy, and the other one concerning security. In particular, we focus on a practical analysis of GAP, given that it is the foundation of several tracing apps. We demonstrate that in real-world scenarios the current GAP design is vulnerable to (i) profiling and possibly de-anonymizing infected persons, and (ii) relay-based wormhole attacks that principally can generate fake contacts with the potential of significantly affecting the accuracy of an app-based contact tracing system. For both types of attack, we have built tools that can be easily used on mobile phones or Raspberry Pis (e.g., Bluetooth sniffers). We hope that our findings provide valuable input in the process of testing and certifying contact tracing apps, e.g., as planned for the German Corona-Warn-App, ultimately guiding improvements for secure and privacy-preserving design and implementation of digital contact tracing systems.}, author = {Baumgärtner, Lars and Dmitrienko, Alexandra and Freisleben, Bernd and Gruler, Alexander and Höchst, Jonas and Kühlberg, Joshua and Mezini, Mira and Miettinen, Markus and Muhamedagic, Anel and Nguyen, Thien Duc and Penning, Alvar and Pustelnik, Dermot Frederik and Roos, Filipp and Sadeghi, Ahmad-Reza and Schwarz, Michael and Uhl, Christian}, journal = {ArXiv}, keywords = {sss-group}, month = {06}, title = {Mind the GAP: Security & Privacy Risks of Contact Tracing Apps}, year = 2020 }
• 
  A Review of Serverless Use Cases and their Characteristics S. Eismann; J. Scheuner; E. van Eyk; M. Schwinger; J. Grohmann; N. Herbst; C. Abad; A. Iosup; (2020).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The serverless computing paradigm promises many desirable properties for cloud applications - low-cost, fine-grained deployment, and management-free operation. Consequently, the paradigm has underwent rapid growth: there currently exist tens of serverless platforms and all global cloud providers host serverless operations. To help tune existing platforms, guide the design of new serverless approaches, and overall contribute to understanding this paradigm, in this work we present a long-term, comprehensive effort to identify, collect, and characterize serverless use cases. We survey 89 use cases, sourced from white and grey literature, and from consultations with experts in areas such as scientific computing. We study each use case using 24 characteristics, including general aspects, but also workload, application, and requirements. When the use cases employ workflows, we further analyze their characteristics. Overall, we hope our study will be useful for both academia and industry, and encourage the community to further share and communicate their use cases.

  @techreport{eismann2020review, abstract = {The serverless computing paradigm promises many desirable properties for cloud applications - low-cost, fine-grained deployment, and management-free operation. Consequently, the paradigm has underwent rapid growth: there currently exist tens of serverless platforms and all global cloud providers host serverless operations. To help tune existing platforms, guide the design of new serverless approaches, and overall contribute to understanding this paradigm, in this work we present a long-term, comprehensive effort to identify, collect, and characterize serverless use cases. We survey 89 use cases, sourced from white and grey literature, and from consultations with experts in areas such as scientific computing. We study each use case using 24 characteristics, including general aspects, but also workload, application, and requirements. When the use cases employ workflows, we further analyze their characteristics. Overall, we hope our study will be useful for both academia and industry, and encourage the community to further share and communicate their use cases.}, author = {Eismann, Simon and Scheuner, Joel and van Eyk, Erwin and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina and Iosup, Alexandru}, keywords = {t_techreport}, month = {06}, publisher = {SPEC RG}, title = {A Review of Serverless Use Cases and their Characteristics}, year = 2020 }
• 
  Towards a Self-Aware Prediction of Critical States. M. Züfle; in Organic Computing: Doctoral Dissertation Colloquium 2020, S. Tomforde, C. Krupitzer (eds.) (2020).

  (to appear)
  • [ BibTeX ]
  @incollection{zufle2020towards, author = {Z{\"u}fle, Marwin}, booktitle = {Organic Computing: Doctoral Dissertation Colloquium 2020}, editor = {Tomforde, Sven and Krupitzer, Christian}, keywords = {t_bookchapter}, month = {07}, note = {(to appear)}, title = {Towards a Self-Aware Prediction of Critical States}, year = 2020 }
• 
  Evaluating the Privacy of Contact Discovery. C. Sendner; Thesis; University of Würzburg. (2020, July).
  • [ Abstract ]
  • [ BibTeX ]
  Currently deployed contact discovery of mobile messengers is based on the transmission of phone numbers to the service provider. This information is private and anonymized by hashing them. In this work, we show, that this anonymization is pseudo-anonymous and can easily be broken by an attacker. For that, we develop two hash reversal techniques: one using brute-force approach and another one using look-up databases. We provide generic architectures for each of the ap- proaches. Additionally, we provide and compare two instantiations for each. Furthermore, we evaluate and compare them to the third approach based on rainbow tables. The evaluation shows near instant lookup-times of under 0.1 ms using in-memory lookup databases, this approach is however costly in terms of memory – it would require over 10 TB RAM, which would be difficult to obtain. Our brute-force approach shows an astonishing performance, being able to reverse any mobile number in under 100 seconds using consumer-level hardware. The rainbow tables produce lookup-times of 4.5 minutes with a success rate of over 99.99%. The results of our evaluation demonstrate, that hash reversals of mobile phone numbers are practical and near instant. Thus, an attacker can easily reverse hash digests of mobile phone numbers and de-anonymize personally identifiable information – like phone numbers transmitted to the service provider of mobile messenger apps.

  @mastersthesis{sendner2020contactdiscovery, abstract = {Currently deployed contact discovery of mobile messengers is based on the transmission of phone numbers to the service provider. This information is private and anonymized by hashing them. In this work, we show, that this anonymization is pseudo-anonymous and can easily be broken by an attacker. For that, we develop two hash reversal techniques: one using brute-force approach and another one using look-up databases. We provide generic architectures for each of the ap- proaches. Additionally, we provide and compare two instantiations for each. Furthermore, we evaluate and compare them to the third approach based on rainbow tables. The evaluation shows near instant lookup-times of under 0.1 ms using in-memory lookup databases, this approach is however costly in terms of memory – it would require over 10 TB RAM, which would be difficult to obtain. Our brute-force approach shows an astonishing performance, being able to reverse any mobile number in under 100 seconds using consumer-level hardware. The rainbow tables produce lookup-times of 4.5 minutes with a success rate of over 99.99%. The results of our evaluation demonstrate, that hash reversals of mobile phone numbers are practical and near instant. Thus, an attacker can easily reverse hash digests of mobile phone numbers and de-anonymize personally identifiable information – like phone numbers transmitted to the service provider of mobile messenger apps.}, author = {Sendner, Christoph}, keywords = {sss-group}, month = {07}, school = {University of W{\"u}rzburg}, title = {Evaluating the Privacy of Contact Discovery}, type = {Master Thesis}, year = 2020 }

• 
  TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research. S. Eismann; (2019, February).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed micro-service applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Unfortunately, existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. In this talk, we introduce the TeaStore, a state-of-the-art test and reference application with a micro-service architecture. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. Based on three experiments, we demonstrate TeaStore's use in three different research contexts: performance modeling, cloud resource management, and energy efficiency analysis.

  @misc{talk-Eismann-SE2019, abstract = {Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed micro-service applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Unfortunately, existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. In this talk, we introduce the TeaStore, a state-of-the-art test and reference application with a micro-service architecture. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. Based on three experiments, we demonstrate TeaStore's use in three different research contexts: performance modeling, cloud resource management, and energy efficiency analysis.}, author = {Eismann, Simon}, howpublished = {Multikonferenz Software Engineering & Management 2019}, keywords = {Cloud}, month = {02}, title = {{TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research}}, year = 2019 }
• 
  Measuring, Rating, and Predicting the Energy Efficiency of Servers. J. von Kistowski; Thesis; University of Würzburg, Germany. (2019, March).
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Kistowski2019-phd, author = {von Kistowski, J{\'o}akim}, keywords = {Power}, month = {03}, school = {University of W{\"u}rzburg, Germany}, title = {Measuring, Rating, and Predicting the Energy Efficiency of Servers}, year = 2019 }
• 
  Integrating Statistical Response Time Models in Architectural Performance Models. S. Eismann; J. Grohmann; J. Walter; J. von Kistowski; S. Kounev; in Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA) (2019). 71–80.

  Acceptance Rate: 21,9% (21/96)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.

  @inproceedings{EiGrWaKiKo2019-ICSA-Integrating, abstract = {Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.}, author = {Eismann, Simon and Grohmann, Johannes and Walter, J{\"u}rgen and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA)}, keywords = {PRISMA}, month = {03}, note = {Acceptance Rate: 21,9\% (21/96)}, pages = {71–80}, publisher = {IEEE}, title = {{Integrating Statistical Response Time Models in Architectural Performance Models}}, year = 2019 }
• 
  SPEC Research Group Newsletter, vol. 2 no. 4. S. Kounev; A. van Hoorn; A. Bauer; N. Herbst; (2019, April).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-8, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{\'e} and Herbst, Nikolas}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 2 no. 4}}, year = 2019 }
• 
  Please Obey My Plan: How to Optimise Decentralised Self-Adaptive Systems. V. Lesch; C. Krupitzer; S. Tomforde; (2019, April).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Lesch-SelPhyS19, author = {Lesch, Veronika and Krupitzer, Christian and Tomforde, Sven}, howpublished = {Talk at Workshop on Self-Awareness in Cyber-Physical Systems 2019}, keywords = {Optimization}, month = {04}, title = {{Please Obey My Plan: How to Optimise Decentralised Self-Adaptive Systems}}, year = 2019 }
• 
  Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud. F. Metzger; T. Hoßfeld; A. Bauer; S. Kounev; P. E. Heegaard; in Proceedings of the IEEE (2019). 107(4) 679–694.
  • [ BibTeX ]
  • [ Download ]
  @article{MeHoBaKoHe2019-PIEEE-IoTTraffic, author = {Metzger, Florian and Ho{\ss}feld, Tobias and Bauer, Andr{\'e} and Kounev, Samuel and Heegaard, Poul. E.}, journal = {Proceedings of the IEEE}, keywords = {t_journalmagazine}, month = {04}, number = 4, pages = {679-694}, title = {{Modeling of Aggregated IoT Traffic and Its Application to an IoT Cloud}}, volume = 107, year = 2019 }
• 
  Methodological Principles for Reproducible Performance Evaluation in Cloud Computing - A SPEC Research Technical Report. (SPEC-RG-2019-04), A. V. Papadopoulos; L. Versluis; A. Bauer; N. Herbst; J. von Kistowski; A. Ali-Eldin; C. Abad; J. N. Amaral; P. Tuma; A. Iosup; (2019).
  • [ BibTeX ]
  • [ URL ]
  @techreport{PaVeBaHeKiEAAbAmTuIo2019-SPEC-CloudPerformance, author = {Papadopoulos, Alessandro Vittorio and Versluis, Laurens and Bauer, Andr{\'e} and Herbst, Nikolas and von Kistowski, J{\'o}akim and Ali-Eldin, Ahmed and Abad, Cristina and Amaral, J. Nelson and Tuma, Petr and Iosup, Alexandru}, institution = {SPEC Research Group — Cloud Working Group, Standard Performance Evaluation Corporation (SPEC)}, keywords = {SPEC}, month = {04}, number = {SPEC-RG-2019-04}, title = {{Methodological Principles for Reproducible Performance Evaluation in Cloud Computing - A SPEC Research Technical Report}}, year = 2019 }
• 
  A Concept for an Adaptive Communication Middleware for Car-2-Cloud Applications. S. Herrnleben; (2019, April).
  • [ BibTeX ]
  • [ Download ]
  @misc{He2019-SelPhyS-Car2Cloud, author = {Herrnleben, Stefan}, howpublished = {Talk at Workshop on Self-Awareness in Cyber-Physical Systems 2019}, keywords = {t_workshop}, month = {04}, title = {{A Concept for an Adaptive Communication Middleware for Car-2-Cloud Applications}}, year = 2019 }
• 
  Die Mehrbelastung der Sicherheit im IoT mit neuen Techniken bewältigen. J. Stoll; L. Iffländer; (2019, April).
  • [ BibTeX ]
  @misc{talk-StIf-SDN-buildingIoT, author = {Stoll, Jonathan and Iffländer, Lukas}, howpublished = {building IoT 2019}, keywords = {descartes}, month = {04}, title = {Die Mehrbelastung der Sicherheit im IoT mit neuen Techniken bewältigen}, year = 2019 }
• 
  Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field. A. Bauer; N. Herbst; S. Spinner; A. Ali-Eldin; S. Kounev; in IEEE Transactions on Parallel and Distributed Systems (2019). 30(4) 800–813.
  • [ BibTeX ]
  • [ Download ]
  @article{BaHeSpAEKo2019-TPDS-ChameleonAutoscaler, author = {Bauer, Andr{\'e} and Herbst, Nikolas and Spinner, Simon and Ali-Eldin, Ahmed and Kounev, Samuel}, journal = {IEEE Transactions on Parallel and Distributed Systems}, keywords = {Self-adaptive-systems}, month = {04}, number = 4, pages = {800–813}, publisher = {IEEE}, title = {{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}}, volume = 30, year = 2019 }
• 
  Improving the Energy Efficiency of IoT-Systems and its Software. N. Schmitt; in Organic Computing: Doctoral Dissertation Colloquium 2018, S. Tomforde, B. Sick (eds.) (2019).
  • [ BibTeX ]
  • [ Download ]
  @incollection{Sc2015-EEIoTSoftware, author = {Schmitt, Norbert}, booktitle = {{Organic Computing: Doctoral Dissertation Colloquium 2018}}, editor = {Tomforde, Sven and Sick, Bernhard}, keywords = {Power}, month = {04}, publisher = {kassel university press GmbH}, title = {{Improving the Energy Efficiency of IoT-Systems and its Software}}, year = 2019 }
• 
  Beyond position-awareness -- Extending a self-adaptive fall detection system. C. Krupitzer; T. Sztyler; J. Edinger; M. Breitbach; H. Stuckenschmidt; C. Becker; in Pervasive and Mobile Computing (2019). 58
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Ambient Assisted Living using mobile device sensors is an active area of research in pervasive computing. Multiple approaches have shown that wearable sensors perform very well and distinguish falls reliably from Activities of Daily Living. However, these systems are tested in a controlled environment and are optimized for a given set of sensor types, sensor positions, and subjects. We propose a self-adaptive pervasive fall detection approach that is robust to the heterogeneity of real life situations. Using the data of four publicly available datasets, we show that our system is not only robust regarding the different dimensions of heterogeneity, but also adapts autonomously to spontaneous changes in the sensor’s position at runtime. In this paper, we extend our self-adaptive fall detection system with (i) additional algorithms for fall detection, (ii) an approach for cross-positional sensor fusion, (iii) a fall detection approach that relies on outlier detection, and (iv) a smart fall alert. Additionally, we present implementation and evaluation of these extensions.

  @article{KRUPITZER2019101026, abstract = {Ambient Assisted Living using mobile device sensors is an active area of research in pervasive computing. Multiple approaches have shown that wearable sensors perform very well and distinguish falls reliably from Activities of Daily Living. However, these systems are tested in a controlled environment and are optimized for a given set of sensor types, sensor positions, and subjects. We propose a self-adaptive pervasive fall detection approach that is robust to the heterogeneity of real life situations. Using the data of four publicly available datasets, we show that our system is not only robust regarding the different dimensions of heterogeneity, but also adapts autonomously to spontaneous changes in the sensor’s position at runtime. In this paper, we extend our self-adaptive fall detection system with (i) additional algorithms for fall detection, (ii) an approach for cross-positional sensor fusion, (iii) a fall detection approach that relies on outlier detection, and (iv) a smart fall alert. Additionally, we present implementation and evaluation of these extensions.}, author = {Krupitzer, Christian and Sztyler, Timo and Edinger, Janick and Breitbach, Martin and Stuckenschmidt, Heiner and Becker, Christian}, journal = {Pervasive and Mobile Computing}, keywords = {Self-adaptive-systems}, title = {Beyond position-awareness – Extending a self-adaptive fall detection system}, volume = 58, year = 2019 }
• 
  Towards Adaptive Car-to-Cloud Communication. S. Herrnleben; M. Pfannemüller; C. Krupitzer; S. Kounev; M. Segata; F. Fastnacht; M. Nigmann; in Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Recent developments in communication technology have led to cloud resources becoming ubiquitous. These resources enable many new applications by offering computational power for remote embedded devices. In combination with advances in the area of smart driving, this seems to be especially beneficial for applications such as remote maintenance of vehicles or integration with smart city services. As autonomous driving continues to gain traction, Car-to-Cloud communication can support transferring collected data to the cloud, e.g., for dynamic learning of new map information. Additionally, passengers can benefit from novel entertainment services. All these developments require a stable connection between a mobile vehicle and the cloud resources. In this vision paper, we survey Car-to-Cloud communication applications. Based on the analysis of the varying requirements for these applications, we formulate research questions and challenges. Further, we discuss how these challenges can be addressed by means of an adaptive Car-to-Cloud communication middleware. We conclude with an overview on our activities in this area and an outlook on our planned future work on adaptive communication.

  @inproceedings{HePfKrKoSeFaNi2019-PerAwareCity-carToCloudCommunication, abstract = {Recent developments in communication technology have led to cloud resources becoming ubiquitous. These resources enable many new applications by offering computational power for remote embedded devices. In combination with advances in the area of smart driving, this seems to be especially beneficial for applications such as remote maintenance of vehicles or integration with smart city services. As autonomous driving continues to gain traction, Car-to-Cloud communication can support transferring collected data to the cloud, e.g., for dynamic learning of new map information. Additionally, passengers can benefit from novel entertainment services. All these developments require a stable connection between a mobile vehicle and the cloud resources. In this vision paper, we survey Car-to-Cloud communication applications. Based on the analysis of the varying requirements for these applications, we formulate research questions and challenges. Further, we discuss how these challenges can be addressed by means of an adaptive Car-to-Cloud communication middleware. We conclude with an overview on our activities in this area and an outlook on our planned future work on adaptive communication.}, address = {Kyoto, Japan}, author = {Herrnleben, Stefan and Pfannem{\"u}ller, Martin and Krupitzer, Christian and Kounev, Samuel and Segata, Michele and Fastnacht, Felix and Nigmann, Magnus}, booktitle = {Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)}, keywords = {Intelligent_Transportation_Systems}, title = {Towards Adaptive {Car-to-Cloud} Communication}, year = 2019 }
• 
  Performance-feedback autoscaling with budget constraints for cloud-based workloads of workflows. A. Ilyushkin; A. Bauer; A. V. Papadopoulos; E. Deelman; A. Iosup; in arXiv preprint arXiv:1905.10270 (2019).
  • [ BibTeX ]
  • [ Download ]
  @article{IlBaPaDeIo-Tech-BudgetAS, author = {Ilyushkin, Alexey and Bauer, Andr{\'e} and Papadopoulos, Alessandro V and Deelman, Ewa and Iosup, Alexandru}, journal = {arXiv preprint arXiv:1905.10270}, keywords = {budget}, title = {Performance-feedback autoscaling with budget constraints for cloud-based workloads of workflows}, year = 2019 }
• 
  A survey on adaptive authentication. P. Arias-Cabarcos; C. Krupitzer; C. Becker; in ACM Computing Surveys (2019). 52 Art. 80.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{ArKrBe2019_CSUR_surveyAdaptiveAuthentication, address = {New York, NY}, author = {Arias-Cabarcos, Patricia and Krupitzer, Christian and Becker, Christian}, journal = {ACM Computing Surveys}, keywords = {Self-adaptive-systems}, pages = {Art. 80}, publisher = {ACM Press}, title = {A survey on adaptive authentication}, volume = 52, year = 2019 }
• 
  How is Performance Addressed in DevOps?. C.-P. Bezemer; S. Eismann; V. Ferme; J. Grohmann; R. Heinrich; P. Jamshidi; W. Shang; A. van Hoorn; M. Villavicencio; J. Walter; F. Willnecker; in Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (2019). 45–50.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  DevOps is a modern software engineering paradigm that is gaining widespread adoption in industry. The goal of DevOps is to bring software changes into production with a high frequency and fast feedback cycles. This conflicts with software quality assurance activities, particularly with respect to performance. For instance, performance evaluation activities --- such as load testing --- require a considerable amount of time to get statistically significant results. We conducted an industrial survey to get insights into how performance is addressed in industrial DevOps settings. In particular, we were interested in the frequency of executing performance evaluations, the tools being used, the granularity of the obtained performance data, and the use of model-based techniques. The survey responses, which come from a wide variety of participants from different industry sectors, indicate that the complexity of performance engineering approaches and tools is a barrier for wide-spread adoption of performance analysis in DevOps. The implication of our results is that performance analysis tools need to have a short learning curve, and should be easy to integrate into the DevOps pipeline in order to be adopted by practitioners.

  @inproceedings{BeEiFeGrRhJaShHoViWaWi2019-ICPE-DevOpsSurvey, abstract = {DevOps is a modern software engineering paradigm that is gaining widespread adoption in industry. The goal of DevOps is to bring software changes into production with a high frequency and fast feedback cycles. This conflicts with software quality assurance activities, particularly with respect to performance. For instance, performance evaluation activities — such as load testing — require a considerable amount of time to get statistically significant results. We conducted an industrial survey to get insights into how performance is addressed in industrial DevOps settings. In particular, we were interested in the frequency of executing performance evaluations, the tools being used, the granularity of the obtained performance data, and the use of model-based techniques. The survey responses, which come from a wide variety of participants from different industry sectors, indicate that the complexity of performance engineering approaches and tools is a barrier for wide-spread adoption of performance analysis in DevOps. The implication of our results is that performance analysis tools need to have a short learning curve, and should be easy to integrate into the DevOps pipeline in order to be adopted by practitioners.}, address = {New York, NY, USA}, author = {Bezemer, Cor{-}Paul and Eismann, Simon and Ferme, Vincenzo and Grohmann, Johannes and Heinrich, Robert and Jamshidi, Pooyan and Shang, Weiyi and van Hoorn, Andr{{\'e}} and Villavicencio, M{{\'o}}nica and Walter, J{{\"u}}rgen and Willnecker, Felix}, booktitle = {Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering}, keywords = {SPEC}, pages = {45–50}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '19}, title = {{How is Performance Addressed in DevOps?}}, year = 2019 }
• 
  Performance Influence of Security Function Chain Ordering. L. Iffländer; N. Fella; in Companion of the 2019 ACM/SPEC International Conference on Performance Engineering (2019). 45–46.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In modern days security systems often reach their performance peak and limit the protected application. Utilizing the available resources for security more efficiently is becoming more critical. In this paper, we introduce the claim, that no static security function chain is optimal in every situation. First experiments prove our claim.

  @inproceedings{Ifflaender2019_PerformanceInfluenceofSecurityFunctionChainOrdering, abstract = {In modern days security systems often reach their performance peak and limit the protected application. Utilizing the available resources for security more efficiently is becoming more critical. In this paper, we introduce the claim, that no static security function chain is optimal in every situation. First experiments prove our claim.}, address = {New York, NY, USA}, author = {Iffländer, Lukas and Fella, Nicolas}, booktitle = {Companion of the 2019 ACM/SPEC International Conference on Performance Engineering}, keywords = {Self-adaptive-systems}, pages = {45–46}, publisher = {ACM}, series = {ICPE '19}, title = {Performance Influence of Security Function Chain Ordering}, year = 2019 }
• 
  Measuring the Energy Efficiency of Transactional Loads on GPGPU. J. von Kistowski; J. Pais; T. Wahl; K.-D. Lange; H. Block; J. Beckett; S. Kounev; in Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering (2019).

  Best Industry Paper Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4% regarding power consumption.

  @inproceedings{KiPaWaLaBlBeKo2019-ICPE-GPGPUPower, abstract = {{General Purpose Graphics Processing Units (GPGPUs) are becoming more and more common in current servers and data centers, which in turn consume a significant amount of electrical power. Measuring and benchmarking this power consumption is important as it helps with optimization and selection of these servers. However, benchmarking and comparing the energy efficiency of GPGPU workloads is challenging as standardized workloads are rare and standardized power and efficiency measurement methods and metrics do not exist. In addition, not all GPGPU systems run at maximum load all the time. Systems that are utilized in transactional, request driven workloads, for example, can run at lower utilization levels. Existing benchmarks for GPGPU systems primarily consider performance and are intended only to run at maximum load. They do not measure performance or energy efficiency at other loads. In turn, server energy-efficiency benchmarks that consider multiple load levels do not address GPGPUs. This paper introduces a measurement methodology for servers with GPGPU accelerators that considers multiple load levels for transactional workloads. The methodology also addresses verifiability of results in order to achieve comparability of different device solutions. We analyze our methodology on three different systems with solutions from two different accelerator vendors. We investigate the efficacy of different methods of load levels scaling and our methodology's reproducibility. We show that the methodology is able to produce consistent and reproducible results with a maximum coefficient of variation of 1.4\% regarding power consumption.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Pais, Johann and Wahl, Tobias and Lange, Klaus-Dieter and Block, Hansfried and Beckett, John and Kounev, Samuel}, booktitle = {Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering}, keywords = {Power}, note = {{<b>Best Industry Paper Award</b>}}, publisher = {ACM}, series = {ICPE '19}, title = {{Measuring the Energy Efficiency of Transactional Loads on GPGPU}}, year = 2019 }
• 
  Measuring and Rating the Energy-Efficiency of Servers. J. von Kistowski; K.-D. Lange; J. A. Arnold; J. Beckett; H. Block; M. Tricker; S. Sharma; J. Pais; S. Kounev; in Future Generation Computer Systems (2019). 100 579–589.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.

  @article{KiLaArBeBlTrShPaKo-FGCS2019-ServerEfficiency, abstract = {Data centers and servers consume a significant amount of electrical power. As future data centers increase in size, it becomes increasingly important to be able to select servers based on their energy efficiency. Rating the efficiency of servers is challenging, as it depends on the software executed on that server and on its load profile. To account for this, a measurement and rating methodology for servers must make use of both realistic and varying workloads. Existing energy-efficiency benchmarks either run standardized application loads at multiple load levels or multiple micro-benchmarks at full load. This does not enable a full analysis of system behavior as the energy efficiency for different kinds of software at low load levels remains unknown. This article introduces a measurement methodology and metrics for energy-efficiency rating of servers. The methodology defines system setup and instrumentation for reproducible results. It is designed to use multiple, specifically chosen workloads at different load levels for a full system characterization. All partial results are aggregated in a final metric that can be used by regulators to define server-labeling criteria. The methodology and workloads have been implemented in the standardized Server Efficiency Rating Tool. We show the applicability and use of the measurement methodology specifically considering its reproducibility, fairness, and relevance. A reproducibility analysis shows that efficiency scores vary with a maximum coefficient of 1.04\% for repeated experiments. In addition, we evaluate the proposed metrics by investigating their energy-efficiency rating on a set of 385 different servers and show the relevance of the selected workloads by analyzing relationships to real-world applications.}, author = {von Kistowski, J{\'o}akim and Lange, Klaus-Dieter and Arnold, Jeremy A. and Beckett, John and Block, Hansfried and Tricker, Mike and Sharma, Sanjay and Pais, Johann and Kounev, Samuel}, journal = {Future Generation Computer Systems}, keywords = {Power}, pages = {579 - 589}, title = {{Measuring and Rating the Energy-Efficiency of Servers}}, volume = 100, year = 2019 }
• 
  Automation in Software Performance Engineering Based on a Declarative Specification of Concerns. J. C. Walter; Thesis; Universität Würzburg. (2019).
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Walter2018-phd, author = {Walter, J{\"u}rgen Christian}, keywords = {PMX}, school = {Universit{\"a}t W{\"u}rzburg}, title = {Automation in Software Performance Engineering Based on a Declarative Specification of Concerns}, type = {doctoralthesis}, year = 2019 }
• 
  Online model learning for self-aware computing infrastructures. S. Spinner; J. Grohmann; S. Eismann; S. Kounev; in Journal of Systems and Software (2019). 147 1–16.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.

  @article{SpGrEiKo2019-JSS-ModelLearning, abstract = {Performance models are valuable and powerful tools for performance prediction. However, the creation of performance models usually requires significant manual effort. Furthermore, as the modeled structures are subject to frequent change in modern infrastructures, such performance models need to be adapted as well. We therefore propose a reference architecture for online model learning in virtualized environments, which enables the automatic extraction of the aforementioned performance models. We follow an agent-based approach, which enables us to incorporate the extraction of information about the application structure as well as the virtualization structures present in modern computing centers. Our evaluation shows that our collaborating agents are able to reduce the manual effort of performance model extraction by 85.4%. The resulting performance model is able to predict the system utilization with an absolute error of less than 4% and the end-to-end response time with a relative error of less than 21%.}, author = {Spinner, Simon and Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, journal = {Journal of Systems and Software}, keywords = {Virtualization}, pages = {1–16}, title = {Online model learning for self-aware computing infrastructures}, volume = 147, year = 2019 }
• 
  Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning. J. Grohmann; P. K. Nicholson; J. O. Iglesias; S. Kounev; D. Lugones; in Proceedings of the 20th International Middleware Conference (2019). 149–162.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics -- while still fulfilling application KPIs -- to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload.

  @inproceedings{GrNiIgKoLu-MIDDLEWARE2019-Monitorless, abstract = {Today, software operation engineers rely on application key performance indicators (KPIs) for sizing and orchestrating cloud resources dynamically. KPIs are monitored to assess the achievable performance and to configure various cloud-specific parameters such as flavors of instances and autoscaling rules, among others. Usually, keeping KPIs within acceptable levels requires application expertise which is expensive and can slow down the continuous delivery of software. Expertise is required because KPIs are normally based on application-specific quality-of-service metrics, like service response time and processing rate, instead of generic platform metrics, like those typical across various environments (e.g., CPU and memory utilization, I/O rate, etc.)In this paper, we investigate the feasibility of outsourcing the management of application performance from developers to cloud operators. In the same way that the serverless paradigm allows the execution environment to be fully managed by a third party, we discuss a monitorless model to streamline application deployment by delegating performance management. We show that training a machine learning model with platform-level data, collected from the execution of representative containerized services, allows inferring application KPI degradation. This is an opportunity to simplify operations as engineers can rely solely on platform metrics – while still fulfilling application KPIs – to configure portable and application agnostic rules and other cloud-specific parameters to automatically trigger actions such as autoscaling, instance migration, network slicing, etc.Results show that monitorless infers KPI degradation with an accuracy of 97% and, notably, it performs similarly to typical autoscaling solutions, even when autoscaling rules are optimally tuned with knowledge of the expected workload.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Nicholson, Patrick K. and Iglesias, Jesus Omana and Kounev, Samuel and Lugones, Diego}, booktitle = {Proceedings of the 20th International Middleware Conference}, keywords = {Virtualization}, pages = {149–162}, publisher = {Association for Computing Machinery (ACM)}, series = {Middleware '19}, title = {Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning}, year = 2019 }
• 
  Predicting Server Power Consumption from Standard Rating Results. J. von Kistowski; J. Grohmann; N. Schmitt; S. Kounev; in Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering (2019). 301–312.

  Full Paper Acceptance Rate: 18.6% (13/70)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.

  @inproceedings{KiGrScKo2019-ICPE-PowerPrediction, abstract = {Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Grohmann, Johannes and Schmitt, Norbert and Kounev, Samuel}, booktitle = {Proceedings of the 19th ACM/SPEC International Conference on Performance Engineering}, keywords = {Power}, note = {{Full Paper Acceptance Rate: 18.6\% (13/70)}}, pages = {301–312}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '19}, title = {{Predicting Server Power Consumption from Standard Rating Results}}, year = 2019 }
• 
  Hands Off my Database: Ransomware Detection in Databases through Dynamic Analysis of Query Sequences. L. Iffländer; A. Dmitrienko; C. Hagen; M. Jobst; S. Kounev; in ArXiv | arXiv:1907.06775v1 (2019).
  • [ BibTeX ]
  • [ URL ]
  @article{DBLP:journals/corr/abs-1907-06775, author = {Iffl{\"{a}}nder, Lukas and Dmitrienko, Alexandra and Hagen, Christoph and Jobst, Michael and Kounev, Samuel}, journal = {ArXiv | arXiv:1907.06775v1}, keywords = {ransomware}, title = {Hands Off my Database: Ransomware Detection in Databases through Dynamic Analysis of Query Sequences}, year = 2019 }
• 
  Performance Oriented Dynamic Bypassing for Intrusion Detection Systems. L. Iffländer; J. Stoll; N. Rawtani; V. Lesch; K.-D. Lange; S. Kounev; in Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (2019). 159–166.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

  @inproceedings{Ifflaender2019_PerformanceOrientedDynamicBypassingforIntrusionDetectionSystems, abstract = {Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.}, address = {New York, NY, USA}, author = {Iffländer, Lukas and Stoll, Jonathan and Rawtani, Nishant and Lesch, Veronika and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering}, keywords = {Optimization}, pages = {159–166}, publisher = {ACM}, series = {ICPE '19}, title = {Performance Oriented Dynamic Bypassing for Intrusion Detection Systems}, year = 2019 }
• 
  WheelShare: Crowd-sensed Surface Classification for Accessible Routing. J. Edinger; V. Raychoudhury; A. Hofmann; A. Wachner; C. Becker; C. Krupitzer; in Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Accessible path routing for wheeled mobility is an important problem given the permanent and temporary obstacles in the built environment. Existing research works have focused on identifying several obstacles as well as facilities such as crosswalks with traffic signals using smartphone based sensing or crowd-sourcing and used those knowledge to generate accessible routes. While all those research works have identified the nature of the road or sidewalk surface (even/uneven) as a major accessibility concern, there is no in-depth empirical research regarding how the accessibility varies from surface to surface. In this work, we propose a hybrid system, called WheelShare which can clearly classify (in/)accessible surfaces.We have analyzed vibration (using an aide-propelled manual wheelchair) data from 11 different standard indoor and outdoor surfaces found in the built environment and classified them using WheelShare into five different categories across the accessibility spectrum with an accuracy of up to 95\%.Our objective knowledge of surfaces is then applied to generate an accessible route through the best possible surface depending on user and wheelchair requirements. WheelShare is 1) scalable, as it uses crowd-sensing to collect data, 2) dynamic, as the data gets constantly updated, and 3) objective, as it uses an objective and data-centric approach.

  @inproceedings{EdRaHoWaBeKr2019-TSUM-wheelShareConcept, abstract = {Accessible path routing for wheeled mobility is an important problem given the permanent and temporary obstacles in the built environment. Existing research works have focused on identifying several obstacles as well as facilities such as crosswalks with traffic signals using smartphone based sensing or crowd-sourcing and used those knowledge to generate accessible routes. While all those research works have identified the nature of the road or sidewalk surface (even/uneven) as a major accessibility concern, there is no in-depth empirical research regarding how the accessibility varies from surface to surface. In this work, we propose a hybrid system, called WheelShare which can clearly classify (in/)accessible surfaces.We have analyzed vibration (using an aide-propelled manual wheelchair) data from 11 different standard indoor and outdoor surfaces found in the built environment and classified them using WheelShare into five different categories across the accessibility spectrum with an accuracy of up to 95\%.Our objective knowledge of surfaces is then applied to generate an accessible route through the best possible surface depending on user and wheelchair requirements. WheelShare is 1) scalable, as it uses crowd-sensing to collect data, 2) dynamic, as the data gets constantly updated, and 3) objective, as it uses an objective and data-centric approach.}, address = {Kyoto, Japan}, author = {Edinger, Janick and Raychoudhury, Vaskar and Hofmann, Alexandra and Wachner, Anton and Becker, Christian and Krupitzer, Christian}, booktitle = {Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)}, keywords = {Optimization}, title = {{WheelShare: Crowd-sensed Surface Classification for Accessible Routing}}, year = 2019 }
• 
  Towards a QoS-aware Cyber Physical Networking Middleware Architecture. M. Brinkschulte; C. Becker; C. Krupitzer; in 1st International Workshop on Middleware for Lightweight, Spontaneous Environments (MISE) @ ACM/IFIP/USENIX International Middleware Conference (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{BrBeKr2019-MISE-CPN_Middleware_Vision, address = {Davis, CA, USA}, author = {Brinkschulte, Melanie and Becker, Christian and Krupitzer, Christian}, booktitle = {1st International Workshop on Middleware for Lightweight, Spontaneous Environments (MISE) @ ACM/IFIP/USENIX International Middleware Conference}, keywords = {Self-adaptive-systems}, title = {{Towards a QoS-aware Cyber Physical Networking Middleware Architecture}}, year = 2019 }
• 
  The Organic Computing Doctoral Dissertation Colloquium: Status and Overview in 2019. C. Krupitzer; S. Tomforde; in Proceedings of the INFORMATIK 2019 (Workshop-Beitr{ä}ge) (2019).
  • [ BibTeX ]
  @inproceedings{KrTo2019_Informatik_OC-DDC, author = {Krupitzer, Christian and Tomforde, Sven}, booktitle = {Proceedings of the INFORMATIK 2019 (Workshop-Beitr{\"a}ge)}, keywords = {Self-adaptive-systems}, organization = {Gesellschaft f{\"u}r Informatik eV}, title = {The Organic Computing Doctoral Dissertation Colloquium: Status and Overview in 2019}, year = 2019 }
• 
  Challenges and Approaches: Forecasting for Autonomic Computing. A. Bauer; in Organic Computing: Doctoral Dissertation Colloquium 2018, S. Tomforde, B. Sick (eds.) (2019).
  • [ BibTeX ]
  • [ Download ]
  @incollection{Ba2019-OCDDC-Forecatsing, author = {Bauer, Andr{\'e}}, booktitle = {{Organic Computing: Doctoral Dissertation Colloquium 2018}}, editor = {Tomforde, Sven and Sick, Bernhard}, keywords = {Self-adaptive-systems}, month = {04}, publisher = {kassel university press GmbH}, title = {{Challenges and Approaches: Forecasting for Autonomic Computing}}, year = 2019 }
• 
  Emerging Self-Integration through Coordination of Autonomous Adaptive Systems. V. Lesch; C. Krupitzer; S. Tomforde; in Proceedings of the 4th IEEE International Workshops on Foundations and Applications of Self* Systems, FAS*W@ICAC/SASO 2019 (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{LeKrTo2019_SISSY, author = {Lesch, Veronika and Krupitzer, Christian and Tomforde, Sven}, booktitle = {Proceedings of the 4th {IEEE} International Workshops on Foundations and Applications of Self* Systems, FAS*W@ICAC/SASO 2019}, keywords = {t_workshop}, month = {06}, publisher = {IEEE}, title = {{Emerging Self-Integration through Coordination of Autonomous Adaptive Systems}}, year = 2019 }
• 
  On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework. M. D’Angelo; S. Gerasimou; S. Ghahremani; J. Grohmann; I. Nunes; E. Pournaras; S. Tomforde; in Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (2019). 13–24.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Collective self-adaptive systems (CSAS) are distributed and interconnected systems composed of multiple agents that can perform complex tasks such as environmental data collection, search and rescue operations, and discovery of natural resources. By providing individual agents with learning capabilities, CSAS can cope with challenges related to distributed sensing and decision-making and operate in uncertain environments. This unique characteristic of CSAS enables the collective to exhibit robust behaviour while achieving system-wide and agent-specific goals. Although learning has been explored in many CSAS applications, selecting suitable learning models and techniques remains a significant challenge that is heavily influenced by expert knowledge. We address this gap by performing a multifaceted analysis of existing CSAS with learning capabilities reported in the literature. Based on this analysis, we introduce a 3D framework that illustrates the learning aspects of CSAS considering the dimensions of autonomy, knowledge access, and behaviour, and facilitates the selection of learning techniques and models. Finally, using example applications from this analysis, we derive open challenges and highlight the need for research on collaborative, resilient and privacy-aware mechanisms for CSAS.

  @inproceedings{SEAMS2019-CollectiveLearningSurvey, abstract = {Collective self-adaptive systems (CSAS) are distributed and interconnected systems composed of multiple agents that can perform complex tasks such as environmental data collection, search and rescue operations, and discovery of natural resources. By providing individual agents with learning capabilities, CSAS can cope with challenges related to distributed sensing and decision-making and operate in uncertain environments. This unique characteristic of CSAS enables the collective to exhibit robust behaviour while achieving system-wide and agent-specific goals. Although learning has been explored in many CSAS applications, selecting suitable learning models and techniques remains a significant challenge that is heavily influenced by expert knowledge. We address this gap by performing a multifaceted analysis of existing CSAS with learning capabilities reported in the literature. Based on this analysis, we introduce a 3D framework that illustrates the learning aspects of CSAS considering the dimensions of autonomy, knowledge access, and behaviour, and facilitates the selection of learning techniques and models. Finally, using example applications from this analysis, we derive open challenges and highlight the need for research on collaborative, resilient and privacy-aware mechanisms for CSAS.}, author = {{D'Angelo}, M. and {Gerasimou}, S. and {Ghahremani}, S. and {Grohmann}, J. and {Nunes}, I. and {Pournaras}, E. and {Tomforde}, S.}, booktitle = {Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems}, keywords = {Self-adaptive-systems}, month = {05}, pages = {13–24}, publisher = {IEEE Press}, series = {SEAMS '19}, title = {On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework}, year = 2019 }
• 
  A Modular Simulation Framework for Analyzing Platooning Coordination. C. Krupitzer; V. Lesch; M. Pfannemüller; C. Becker; M. Segata; in Proceedings of the 1st ACM Workshop on Technologies, mOdels, and Protocols for Cooperative Connected Cars (TOP-Cars), Colocated with ACM MobiHoc 2019 (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KrLePfBeSe2019_TOPCars, author = {Krupitzer, Christian and Lesch, Veronika and Pfannem{\"u}ller, Martin and Becker, Christian and Segata, Michele}, booktitle = {Proceedings of the 1st {ACM} Workshop on Technologies, mOdels, and Protocols for Cooperative Connected Cars (TOP-Cars), Colocated with ACM MobiHoc 2019}, keywords = {Intelligent_Transportation_Systems}, month = {07}, publisher = {ACM}, title = {{A Modular Simulation Framework for Analyzing Platooning Coordination}}, year = 2019 }
• 
  Multi-objective Optimisation in Hybrid Collaborating Adaptive Systems. V. Lesch; C. Krupitzer; S. Tomforde; in Proceedings of the 7th edition in the Series on Autonomously Learning and Optimising Systems (SAOS), co-located with 32nd GI/ITG ARCS 2019 (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{LeKrTo2019, author = {Lesch, Veronika and Krupitzer, Christian and Tomforde, Sven}, booktitle = {Proceedings of the 7th edition in the Series on Autonomously Learning and Optimising Systems (SAOS), co-located with 32nd GI/ITG ARCS 2019}, keywords = {Optimization}, month = {05}, publisher = {Gesellschaft fuer Informatik (GI)}, title = {{Multi-objective Optimisation in Hybrid Collaborating Adaptive Systems}}, year = 2019 }
• 
  Exploring the Limitations of Statistical Response Time Models in Architectural Performance Models. S. Trapp; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2019, December).
  • [ BibTeX ]
  @mastersthesis{Trapp2019, address = {Am Hubland, Informatikgeb{\"a}ude, 97074 W{\"u}rzburg, Germany}, author = {Trapp, Simon}, keywords = {descartes}, month = 12, school = {University of W{\"u}rzburg}, title = {Exploring the {L}imitations of {S}tatistical {R}esponse {T}ime {M}odels in {A}rchitectural {P}erformance {M}odels}, type = {{Bachelor Thesis}}, year = 2019 }
• 
  DR.SGX: Automated and Adjustable Side-Channel Protection for SGX using Data Location Randomization. F. Brasser; S. Capkun; A. Dmitrienko; T. Frassetto; K. Kostiainen; A.-R. Sadeghi; in Annual Computer Security Applications Conference (ACSAC) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  Recent research has demonstrated that Intel’s SGX is vulnerable to software-based side-channel attacks. In a common attack, the adversary monitors CPU caches to infer secret-dependent data accesses patterns. Known defenses have major limitations, as they require either error-prone developer assistance, incur extremely high runtime overhead, or prevent only specific attacks.In this paper, we propose data location randomization as a novel defense against side-channel attacks that target data access patterns.Our goal is to break the link between the memory observations by the adversary and the actual data accesses by the victim. We design and implement a compiler-based tool called DR.SGX that instruments the enclave code, permuting data locations at fine granularity.To prevent correlation of repeated memory accesses we periodically re-randomize all enclave data. Our solution requires no developer assistance and strikes the balance between side-channel protection and performance based on an adjustable security parameter.

  @inproceedings{BrCaDmFrKoSa2019, abstract = {Recent research has demonstrated that Intel’s SGX is vulnerable to software-based side-channel attacks. In a common attack, the adversary monitors CPU caches to infer secret-dependent data accesses patterns. Known defenses have major limitations, as they require either error-prone developer assistance, incur extremely high runtime overhead, or prevent only specific attacks.In this paper, we propose data location randomization as a novel defense against side-channel attacks that target data access patterns.Our goal is to break the link between the memory observations by the adversary and the actual data accesses by the victim. We design and implement a compiler-based tool called DR.SGX that instruments the enclave code, permuting data locations at fine granularity.To prevent correlation of repeated memory accesses we periodically re-randomize all enclave data. Our solution requires no developer assistance and strikes the balance between side-channel protection and performance based on an adjustable security parameter.}, author = {Brasser, Ferdinand and Capkun, Srdjan and Dmitrienko, Alexandra and Frassetto, Tommaso and Kostiainen, Kari and Sadeghi, Ahmad-Reza}, booktitle = {Annual Computer Security Applications Conference (ACSAC)}, keywords = {sss-group}, month = 12, publisher = {ACM}, title = {{DR.SGX: Automated and Adjustable Side-Channel Protection for SGX using Data Location Randomization}}, year = 2019 }
• 
  Kaa: Evaluating Elasticity of Cloud-hosted DBMS. D. Seybold; S. Volpert; S. Wesner; A. Bauer; N. Herbst; J. Domaschka; in Proceedings of the 11th IEEE International Conference on Cloud Computing (CloudCom) (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{SeVoWeBaHeDo-CloudCom-Kaa, author = {Seybold, Daniel and Volpert, Simon and Wesner, Stefan and Bauer, Andr{\'e} and Herbst, Nikolas and Domaschka, J{\"o}rg}, booktitle = {Proceedings of the 11th IEEE International Conference on Cloud Computing (CloudCom)}, keywords = {Elasticity}, month = 12, title = {Kaa: Evaluating Elasticity of Cloud-hosted DBMS}, year = 2019 }
• 
  DR.SGX: Automated and Adjustable Side-Channel Protection for SGX using Data Location Randomization F. Brasser; S. Capkun; A. Dmitrienko; T. Frassetto; K. Kostiainen; A.-R. Sadeghi; (2019, December).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Recent research has demonstrated that Intel’s SGX is vulnerable to software-based side-channel attacks. In a common attack, the adversary monitors CPU caches to infer secret-dependent data accesses patterns. Known defenses have major limitations, as they require either error-prone developer assistance, incur extremely high runtime overhead, or prevent only specific attacks.In this paper, we propose data location randomization as a novel defense against side-channel attacks that target data access patterns.Our goal is to break the link between the memory observations by the adversary and the actual data accesses by the victim. We design and implement a compiler-based tool called DR.SGX that instruments the enclave code, permuting data locations at fine granularity.To prevent correlation of repeated memory accesses we periodically re-randomize all enclave data. Our solution requires no developer assistance and strikes the balance between side-channel protection and performance based on an adjustable security parameter.

  @preprint{brasser2019drsgx, abstract = {Recent research has demonstrated that Intel’s SGX is vulnerable to software-based side-channel attacks. In a common attack, the adversary monitors CPU caches to infer secret-dependent data accesses patterns. Known defenses have major limitations, as they require either error-prone developer assistance, incur extremely high runtime overhead, or prevent only specific attacks.In this paper, we propose data location randomization as a novel defense against side-channel attacks that target data access patterns.Our goal is to break the link between the memory observations by the adversary and the actual data accesses by the victim. We design and implement a compiler-based tool called DR.SGX that instruments the enclave code, permuting data locations at fine granularity.To prevent correlation of repeated memory accesses we periodically re-randomize all enclave data. Our solution requires no developer assistance and strikes the balance between side-channel protection and performance based on an adjustable security parameter.}, author = {Brasser, Ferdinand and Capkun, Srdjan and Dmitrienko, Alexandra and Frassetto, Tommaso and Kostiainen, Kari and Sadeghi, Ahmad-Reza}, booktitle = {Annual Computer Security Applications Conference (ACSAC)}, keywords = {sss-group}, month = 12, publisher = {ACM}, title = {{DR.SGX: Automated and Adjustable Side-Channel Protection for SGX using Data Location Randomization}}, year = 2019 }
• 
  Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces. L. Beierlieb; L. Iffländer; A. Milenkoski; C. F. Goncalves; N. Antunes; S. Kounev; in 2019 IEEE International Symposium on Software Reliability Engineering Workshops ({ISSREW}) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.

  @inproceedings{Beierlieb_2019_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces, abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.}, author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Goncalves, Charles F. and Antunes, Nuno and Kounev, Samuel}, booktitle = {2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})}, keywords = {t_workshop}, month = 11, organization = {{IEEE}}, title = {Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces}, year = 2019 }
• 
  Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior. L. Beierlieb; L. Iffländer; S. Kounev; A. Milenkoski; in Proceedings of the 10th Symposium on Software Performance 2019 (SSP’19) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.

  @inproceedings{Beierlieb_2019_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior, abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.}, author = {Beierlieb, Lukas and Iffländer, Lukas and Kounev, Samuel and Milenkoski, Aleksandar}, booktitle = {Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)}, keywords = {t_workshop}, month = 11, title = {Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior}, year = 2019 }
• 
  The SPEC-RG Reference Architecture for FaaS: From Microservices and Containers to Serverless Platforms. E. van Eyk; J. Grohmann; S. Eismann; A. Bauer; L. Versluis; L. Toader; N. Schmitt; N. Herbst; C. L. Abad; A. Iosup; in IEEE Internet Computing (2019). 23(6) 7–18.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Microservices, containers, and serverless computing belong to a trend toward applications composed of many small, self-contained, and automatically managed components. Core to serverless computing, Function-as-a-Service (FaaS) platforms employ state-of-the-art container technology and microservices-based architectures to enable users to manage complex applications without the need for systems-level expertise. Victim of its own success, and partially due to proprietary technology, currently the community has a limited overview of these platforms. To address this, we propose a reference architecture and ecosystem for FaaS platforms. Based on a year-long survey of real-world platforms conducted within the SPEC-RG Cloud Group, we highlight specific components and identify common operational patterns.

  @article{vEGrEiBaVeToScHeAbIo-IC-FaaS, abstract = {Microservices, containers, and serverless computing belong to a trend toward applications composed of many small, self-contained, and automatically managed components. Core to serverless computing, Function-as-a-Service (FaaS) platforms employ state-of-the-art container technology and microservices-based architectures to enable users to manage complex applications without the need for systems-level expertise. Victim of its own success, and partially due to proprietary technology, currently the community has a limited overview of these platforms. To address this, we propose a reference architecture and ecosystem for FaaS platforms. Based on a year-long survey of real-world platforms conducted within the SPEC-RG Cloud Group, we highlight specific components and identify common operational patterns.}, author = {van Eyk, Erwin and Grohmann, Johannes and Eismann, Simon and Bauer, Andr{\'e} and Versluis, Laurens and Toader, Lucian and Schmitt, Norbert and Herbst, Nikolas and Abad, Cristina L. and Iosup, Alexandru}, journal = {IEEE Internet Computing}, keywords = {SPEC}, month = 11, number = 6, pages = {7–18}, publisher = {IEEE}, title = {The SPEC-RG Reference Architecture for FaaS: From Microservices and Containers to Serverless Platforms}, volume = 23, year = 2019 }
• 
  On Learning Parametric Dependencies from Monitoring Data. J. Grohmann; S. Eismann; S. Kounev; in Proceedings of the 10th Symposium on Software Performance 2019 (SSP’19) (2019).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  A common approach to predict system performance are so-called architectural performance models. In these models, parametric dependencies describe the relation between the input parameters of a component and its performance properties and therefore significantly increase the model expressiveness. However, manually modeling parametric dependencies is often infeasible in practice. Existing automated extraction approaches require either application source code or dedicated performance tests, which are not always available. We therefore introduced one approach for identification and one for characterization of parametric dependencies, solely based on run-time monitoring data. In this paper, we propose our idea on combining both techniques in order to create a holistic approach for the identification and characterization of parametric dependencies. Furthermore, we discuss challenges we are currently facing and potential ideas on how to overcome them.

  @inproceedings{GrEiKo2019-SSP-LearningDependencies, abstract = {A common approach to predict system performance are so-called architectural performance models. In these models, parametric dependencies describe the relation between the input parameters of a component and its performance properties and therefore significantly increase the model expressiveness. However, manually modeling parametric dependencies is often infeasible in practice. Existing automated extraction approaches require either application source code or dedicated performance tests, which are not always available. We therefore introduced one approach for identification and one for characterization of parametric dependencies, solely based on run-time monitoring data. In this paper, we propose our idea on combining both techniques in order to create a holistic approach for the identification and characterization of parametric dependencies. Furthermore, we discuss challenges we are currently facing and potential ideas on how to overcome them.}, author = {Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)}, keywords = {t_workshop}, month = 11, title = {{On Learning Parametric Dependencies from Monitoring Data}}, year = 2019 }
• 
  Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques. J. Grohmann; S. Eismann; S. Elflein; M. Mazkatli; J. von Kistowski; S. Kounev; in 2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2019). 309–322.

  Acceptance Rate: 23.8% (29/122)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.

  @inproceedings{GrEiElMaKiKo2019-MASCOTS-DependencyIdentification, abstract = {Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.}, author = {Grohmann, Johannes and Eismann, Simon and Elflein, Sven and Mazkatli, Manar and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {2019 IEEE 27th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {Automated_model_learning}, month = 10, note = {{Acceptance Rate: 23.8% (29/122)}}, pages = {309–322}, publisher = {IEEE Computer Society}, series = {MASCOTS '19}, title = {{Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques}}, year = 2019 }
• 
  Discrete-Time Analysis of the Blockchain Distributed Ledger Technology. S. Geissler; T. Prantl; S. Lange; F. Wamser; T. Hossfeld; in Proceedings of the 31st International Teletraffic Congress (ITC) (2019).
  • [ BibTeX ]
  @inproceedings{GePrLaWaHo2019-ITC-Blockchain, author = {Geissler, Stefan and Prantl, Thomas and Lange, Stanislav and Wamser, Florian and Hossfeld, Tobias}, booktitle = {Proceedings of the 31st International Teletraffic Congress (ITC)}, keywords = {descartes}, month = {08}, title = {Discrete-Time Analysis of the Blockchain Distributed Ledger Technology}, year = 2019 }
• 
  From the Internet-of-Things to the Internet-of-Thinking-Things. S. Kounev; (2019, July).
  • [ BibTeX ]
  @misc{talk-Kounev-InvTUM19, author = {Kounev, Samuel}, howpublished = {Informatik-Sonderkolloquium "Engineering Software for Decentralized Systems", Fakult{\"a}t f{\"u}r Informatik, Technische Universit{\"a}t M{\"u}nchen, Munich, Germany}, keywords = {descartes}, month = {07}, title = {{From the Internet-of-Things to the Internet-of-Thinking-Things}}, year = 2019 }
• 
  Artificial Intelligence in Medicine - From Data Collection to Prediction. M. Züfle; R. Leppich; (2019, November).
  • [ BibTeX ]
  @misc{talk-ZuLe, author = {Z{\"u}fle, Marwin and Leppich, Robert}, howpublished = {Invited Talk at Klinikum N{\"u}rnberg}, keywords = {Sports}, month = 11, title = {{Artificial Intelligence in Medicine - From Data Collection to Prediction}}, year = 2019 }
• 
  Methodological Principles for Reproducible Performance Evaluation in Cloud Computing. A. V. Papadopoulos; L. Versluis; A. Bauer; N. Herbst; J. von Kistowski; A. Ali-Eldin; C. Abad; J. N. Amaral; P. Tuma; A. Iosup; in IEEE Transactions on Software Engineering (2019). 47(8) 1528–1543.
  • [ BibTeX ]
  • [ DOI ]
  • [ Download ]
  @article{PaVeBaHeKiAEAbAmTuIo2019-TSE-MethodologicalPrinciples, author = {Papadopoulos, Alessandro Vittorio and Versluis, Laurens and Bauer, Andr{\'e} and Herbst, Nikolas and von Kistowski, J{\'o}akim and Ali-Eldin, Ahmed and Abad, Cristina and Amaral, J. Nelson and Tuma, Petr and Iosup, Alexandru}, journal = {IEEE Transactions on Software Engineering}, keywords = {SPEC}, month = {07}, number = 8, pages = {1528-1543}, title = {Methodological Principles for Reproducible Performance Evaluation in Cloud Computing}, volume = 47, year = 2019 }
• 
  Autonomic Forecasting Method Selection: Examination and Ways Ahead. M. Züfle; A. Bauer; V. Lesch; C. Krupitzer; N. Herbst; S. Kounev; V. Curtef; in Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{ZuBaLeKrHeKoCu2019-ICAC-Autonomic, author = {Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Lesch, Veronika and Krupitzer, Christian and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 16th {IEEE} International Conference on Autonomic Computing (ICAC)}, keywords = {PRISMA}, month = {06}, publisher = {IEEE}, title = {{Autonomic Forecasting Method Selection: Examination and Ways Ahead}}, year = 2019 }
• 
  Hands Off my Database: Ransomware Detection in Databases through Dynamic Analysis of Query Sequences L. Iffländer; A. Dmitrienko; C. Hagen; M. Jobst; S. Kounev; (2019).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @techreport{Ifflander_Hands_Off_my_Database, author = {Iffländer, Lukas and Dmitrienko, Alexandra and Hagen, Christoph and Jobst, Michael and Kounev, Samuel}, institution = {Universit{\"a}t W{\"u}rzburg}, journal = {arXiv}, keywords = {t_techreport}, month = {07}, title = {Hands Off my Database: Ransomware Detection in Databases through Dynamic Analysis of Query Sequences}, year = 2019 }
• 
  Planning as Optimization: Dynamically Discovering Optimal Configurations for Runtime Situations. E. M. Fredericks; I. Gerostathopoulos; C. Krupitzer; T. Vogel; in Proceedings of the 13th IEEE International Conference on Self-Adaptive and Self-Organizing Systems, SASO 2019 (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{FrGeKrVo2019, author = {Fredericks, Erik M. and Gerostathopoulos, Ilias and Krupitzer, Christian and Vogel, Thomas}, booktitle = {Proceedings of the 13th {IEEE} International Conference on Self-Adaptive and Self-Organizing Systems, {SASO} 2019}, keywords = {Optimization}, month = {06}, publisher = {IEEE}, title = {Planning as Optimization: Dynamically Discovering Optimal Configurations for Runtime Situations}, year = 2019 }
• 
  Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field. A. Bauer; (2019, June).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Bauer-HotCloudPerf2019, author = {Bauer, Andr{\'e}}, howpublished = {Talk at HotCloudperf 2019}, keywords = {Chameleon}, month = {06}, title = {{Chameleon: A Hybrid, Proactive Auto-Scaling Mechanism on a Level-Playing Field}}, year = 2019 }
• 
  Agile Scalability Engineering: The ScrumScale Method. G. Brataas; G. Hanssen; N. Herbst; A. van Hoorn; in IEEE Software (2019). 37(5) 77–85.

  to appear
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  Scalability is a property that must be carefully designed into a system. A case study in the largest Norwegian public portal, Altinn, illustrates how developers and scalability experts improved scalability and spent less time during scalability testing. With minor adjustments to an agile development process, stakeholders spend more up-front time together.

  @article{ScrumScale2019IEEESW, abstract = {Scalability is a property that must be carefully designed into a system. A case study in the largest Norwegian public portal, Altinn, illustrates how developers and scalability experts improved scalability and spent less time during scalability testing. With minor adjustments to an agile development process, stakeholders spend more up-front time together.}, author = {Brataas, Gunnar and Hanssen, Geir and Herbst, Nikolas and van Hoorn, Andr{\'e}}, journal = {IEEE Software}, keywords = {se2}, month = {06}, note = {to appear}, number = 5, pages = {77-85}, publisher = {IEEE}, title = {Agile Scalability Engineering: The ScrumScale Method}, volume = 37, year = 2019 }
• 
  Der Weg vom Internet-der-Dinge zum Internet-der-denkenden-Dinge. S. Kounev; (2019, June).
  • [ BibTeX ]
  @misc{Talk-Ko2019-RotaryClub-SeAC, address = {W{\"u}rzburz, Germany}, author = {Kounev, Samuel}, howpublished = {Rotary Club W{\"u}rzburg, 11.06.2019}, keywords = {descartes}, month = {06}, title = {{Der Weg vom Internet-der-Dinge zum Internet-der-denkenden-Dinge}}, year = 2019 }
• 
  Towards Edge Benchmarking: A Methodology for Characterizing Edge Workloads. K. Tocze; N. Schmitt; I. Brandic; A. Aral; S. Nadjm-Tehrani; in Proceedings of Workshop on Hot Topics in Cloud Computing Performance (HotCloudPerf) as part of FAS*(IEEE ICAC/SASO) conferences companion (2019).
  • [ BibTeX ]
  @inproceedings{ToScBrArNa2019TowardsEdgeBench, author = {Tocze, Klervie and Schmitt, Norbert and Brandic, Ivona and Aral, Atakan and Nadjm-Tehrani, Simin}, booktitle = {Proceedings of Workshop on Hot Topics in Cloud Computing Performance (HotCloudPerf) as part of FAS*(IEEE ICAC/SASO) conferences companion}, keywords = {Power}, month = {06}, publisher = {IEEE}, title = {{Towards Edge Benchmarking: A Methodology for Characterizing Edge Workloads}}, year = 2019 }
• 
  Systematic Search for Optimal Resource Configurations of Distributed Applications. A. Bauer; S. Eismann; J. Grohmann; N. Herbst; S. Kounev; in 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2019). 120–125.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  With the advent of the micro-service paradigm, applications are divided into small, distributed parts. Knowledge of optimal resource configurations of such applications is required both for autonomic resource management as well as its assessment. Due to the high-dimensional search space of all possible configurations, the systematic measuring of the optimal configurations is challenging. To this end, we introduce a search algorithm based on hill-climbing for finding all optimal configurations in a feasible time and integrate it in an existing measuring framework. This approach enables the assessment, comparison and optimization of autonomic resource management approaches for micro-service applications. The evaluation shows that our approach is able to find all optimal configurations in the considered scenarios, while state-of-the-art multi-objective search algorithms do not.

  @inproceedings{BaEiGrHeKo-ICAC-BUNGEE, abstract = {With the advent of the micro-service paradigm, applications are divided into small, distributed parts. Knowledge of optimal resource configurations of such applications is required both for autonomic resource management as well as its assessment. Due to the high-dimensional search space of all possible configurations, the systematic measuring of the optimal configurations is challenging. To this end, we introduce a search algorithm based on hill-climbing for finding all optimal configurations in a feasible time and integrate it in an existing measuring framework. This approach enables the assessment, comparison and optimization of autonomic resource management approaches for micro-service applications. The evaluation shows that our approach is able to find all optimal configurations in the considered scenarios, while state-of-the-art multi-objective search algorithms do not.}, address = {Los Alamitos, CA, USA}, author = {Bauer, André and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}, keywords = {Optimization}, month = {06}, pages = {120–125}, publisher = {IEEE Computer Society}, title = {Systematic Search for Optimal Resource Configurations of Distributed Applications}, year = 2019 }
• 
  Methoden und Messverfahren für Automatisches Skalieren in Elastischen Cloud Umgebungen. N. Herbst; (2019, June).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Herbst-GIAwardNomination-Dagstuhl, author = {Herbst, Nikolas}, howpublished = {GI Dissertationspreis: Seminar f{\"u}r Nominierte Doktoranden, Dagstuhl, Germany, June 28, 2019}, keywords = {t_bookchapter}, month = {06}, title = {{Methoden und Messverfahren f{\"u}r Automatisches Skalieren in Elastischen Cloud Umgebungen}}, year = 2019 }
• 
  Utilizing Clustering to Optimize Resource Demand Estimation Approaches. J. Grohmann; S. Eismann; A. Bauer; M. Zuefle; N. Herbst; S. Kounev; in 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2019). 134–139.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.

  @inproceedings{GrEiBaZuHeKo2019-ICAC-RDEClustering, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andre and Zuefle, Marwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}, keywords = {Optimization}, month = {06}, pages = {134–139}, publisher = {IEEE}, title = {{Utilizing Clustering to Optimize Resource Demand Estimation Approaches}}, year = 2019 }
• 
  Online Power Consumption Estimation for Functions in Cloud Applications. N. Schmitt; L. Iffländer; A. Bauer; S. Kounev; in Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC) (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{ScIfBaKo2019-ICAC-OnlinePowerEstimation, author = {Schmitt, Norbert and Iffländer, Lukas and Bauer, André and Kounev, Samuel}, booktitle = {Proceedings of the 16th IEEE International Conference on Autonomic Computing (ICAC)}, keywords = {Power}, month = {06}, publisher = {IEEE}, title = {Online Power Consumption Estimation for Functions in Cloud Applications}, year = 2019 }
• 
  Best Practices for Time Series Forecasting. A. Bauer; M. Züfle; N. Herbst; S. Kounev; in 2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W) (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{BaZuHeKo2019-ICAC-ForecastingTutorial, author = {Bauer, Andr{\'e} and Z{\"u}fle, Marwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {{2019 IEEE 4th International Workshops on Foundations and Applications of Self* Systems (FAS*W)}}, keywords = {t_tutorial}, month = {06}, title = {{Best Practices for Time Series Forecasting}}, year = 2019 }
• 
  Chamulteon: Coordinated Auto-Scaling of Micro-Services. A. Bauer; V. Lesch; L. Versluis; A. Ilyushkin; N. Herbst; S. Kounev; in Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS) (2019).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{BaLeVeIlHeKo2019-ICDCS-Chamulteon, author = {Bauer, Andr{\'e} and Lesch, Veronika and Versluis, Laurens and Ilyushkin, Alexey and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 39th IEEE International Conference on Distributed Computing Systems (ICDCS)}, keywords = {SPEC}, month = {07}, title = {Chamulteon: Coordinated Auto-Scaling of Micro-Services}, year = 2019 }

• 
  Extending BUNGEE Elasticity Benchmark for Multi-Tier Cloud Applications. A. Bauer; (2018, April).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Bauer-HotCloudPerf2018, author = {Bauer, Andr{\'e}}, howpublished = {Talk at HotCloudperf 2018}, keywords = {BUNGEE}, month = {04}, title = {{Extending BUNGEE Elasticity Benchmark for Multi-Tier Cloud Applications}}, year = 2018 }
• 
  Towards Self-Aware DEVOPS: From Theory to Practice. S. Kounev; (2018, March).
  • [ BibTeX ]
  @misc{Talk-Ko2018-SAP-SeAC_DEVOPS, address = {Potsdam, Germany}, author = {Kounev, Samuel}, howpublished = {Invited Talk, SAP Innovation Center, March 20, 2018, Potsdam, Germany}, keywords = {descartes}, month = {03}, title = {{Towards Self-Aware DEVOPS: From Theory to Practice}}, year = 2018 }
• 
  An Experimental Performance Evaluation of Autoscalers for Complex Workflows. A. Ilyushkin; A. Ali-Eldin; N. Herbst; A. Bauer; A. V. Papadopoulos; D. Epema; A. Iosup; in ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) (2018). 3(2) 8:1–8:32.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{IlAEHeBaPaEpIo2018-TOMPECS-Workflow, address = {New York, NY, USA}, author = {Ilyushkin, Alexey and Ali-Eldin, Ahmed and Herbst, Nikolas and Bauer, Andr{\'e} and Papadopoulos, Alessandro V. and Epema, Dick and Iosup, Alexandru}, journal = {ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS)}, keywords = {SPEC}, month = {04}, number = 2, pages = {8:1–8:32}, publisher = {ACM}, title = {An Experimental Performance Evaluation of Autoscalers for Complex Workflows}, volume = 3, year = 2018 }
• 
  Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time. S. Eismann; J. Walter; J. von Kistowski; S. Kounev; in 2018 IEEE International Conference on Software Architecture (ICSA) (2018). 135–144.

  Acceptance Rate: 25,6% (22/86)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.

  @inproceedings{EiWaKiKo2018-ParametricDependencies, abstract = {Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.}, author = {Eismann, Simon and Walter, J{\"u}rgen and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {2018 IEEE International Conference on Software Architecture (ICSA)}, keywords = {Meta-models}, month = {04}, note = {Acceptance Rate: 25,6\% (22/86)}, pages = {135-144}, title = {{Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time}}, year = 2018 }
• 
  FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds. V. Lesch; A. Bauer; N. Herbst; S. Kounev; in Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018) (2018).

  Full Paper Acceptance Ratio: 23,7%
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{LeBaHeKo-ICPE18-FOX-AS-COST, address = {New York, NY, USA}, author = {Lesch, Veronika and Bauer, Andr{\'e} and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 9th ACM/SPEC International Conference on Performance Engineering (ICPE 2018)}, keywords = {Optimization}, month = {04}, note = {Full Paper Acceptance Ratio: 23,7%}, publisher = {ACM}, title = {{FOX: Cost-Awareness for Autonomic Resource Management in Public Clouds}}, year = 2018 }
• 
  Flexible Performance Predictions at Run-time. S. Eismann; (2018, April).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Performance modeling is a powerful tool to predict the performance of a software system, especially for complex software systems deployed in cloud enviroments. At run-time, these predictions can be used for a variety of purposes. A performance engineer can analyze them to find potential bottlenecks or investigate solutions to an existing performance issue. Alternatively, a design space exploration tool can utilize performance predictions to find an optimal system configuration for a given scenario. A relatively novel use case for performance predictions is proactive auto-scaling. Proactive auto-scaling uses load forecasts to predict load spikes and proactively adapt the system accordingly. Here performance predictions are required to find a suitable system configuration for the incoming load spike. However, all of these use cases have different requirements for the performance predictions in terms of time-to-result, accuracy, and the required metrics. For example during design space exploration accurate predictions of both utilization and response time are required, without major time limitations. An auto-scaler, on the other hand, requires only rough utilization predictions, but with strict time-to-result requirements. A performance engineer requires a balance of accuracy and time-to-result. He is however often only interested in specific metrics, not the performance of the whole system. While a vast number of performance prediction approaches has been proposed, there is as of yet no approach flexible enough to fit all use cases. In this talk, we present our vision for flexible performance predictions at run-time. We envision the usage of one model describing the architecture and performance relevant properties of the system. This model can then be transformed to a number of solution formalisms, such as Queuing Networks (QNs), Layered Queuing Networks (LQNs) or stochastic process models. For these solution formalisms, a series of solution approaches exist, ranging from simple approximations, analytical approaches to full-scale simulations. This ensures that for each use case a suitable solution approach is available without having to maintain multiple models of the system. However, manual selection requires expert knowledge and manual intervention. Therefore, we aim to design an intelligent selection algorithm. This algorithm selects a suitable solution approach for every performance query. We define a performance query as a collection of requested performance metrics, a lower limit for the prediction accuracy and an upper limit for the time-to-result. In order to select a suitable solution approach for a query we first filter out all approaches that are not capable of predicting the requested performance metrics. For example, many analytical approaches are not capable of predicting response time distributions. In a next step, the selection algorithm needs to predict the accuracy and time-to-result for every solution approach. To predict the accuracy we plan to combine expert knowledge about the solvers with smart transformations, that can estimate how well the initial model is represented in the transformation output. For each simplification a transformation makes a number of penalty points, depending the level of simplification is awarded. For example, if a transformation represents loops in a probabilistic fashion only two penalty points per loop would be awarded, while a transformation that ignores passive resources would award a larger number of penalty points for each passive resource in the original model. Additional penalty points are awarded in case the applied solution approach calculates approximations instead of exact results. These penalty scores are determined once using expert knowledge. The resulting penalty score represents an indicator for the expected accuracy of a solution approach. We plan to utilize machine learning approaches to estimate the time-to-result. In general, estimating the time-to-result for performance predictions is challenging. However, we do not need to predict the time-to-result for a previously unseen model. After the inception of the performance model, each solution approach can be benchmarked to generate training data. From this training data, the machine learning approaches can then estimate the time-to-result for a specific request. The accuracy and time-to-result estimations will enable the design of an automated selection algorithm, that selects a suitable solution approach for each performance query. This allows for flexible performance predictions using a single model.

  @misc{talk-Eismann-HotCloudPerf2018, abstract = {Performance modeling is a powerful tool to predict the performance of a software system, especially for complex software systems deployed in cloud enviroments. At run-time, these predictions can be used for a variety of purposes. A performance engineer can analyze them to find potential bottlenecks or investigate solutions to an existing performance issue. Alternatively, a design space exploration tool can utilize performance predictions to find an optimal system configuration for a given scenario. A relatively novel use case for performance predictions is proactive auto-scaling. Proactive auto-scaling uses load forecasts to predict load spikes and proactively adapt the system accordingly. Here performance predictions are required to find a suitable system configuration for the incoming load spike. However, all of these use cases have different requirements for the performance predictions in terms of time-to-result, accuracy, and the required metrics. For example during design space exploration accurate predictions of both utilization and response time are required, without major time limitations. An auto-scaler, on the other hand, requires only rough utilization predictions, but with strict time-to-result requirements. A performance engineer requires a balance of accuracy and time-to-result. He is however often only interested in specific metrics, not the performance of the whole system. While a vast number of performance prediction approaches has been proposed, there is as of yet no approach flexible enough to fit all use cases. In this talk, we present our vision for flexible performance predictions at run-time. We envision the usage of one model describing the architecture and performance relevant properties of the system. This model can then be transformed to a number of solution formalisms, such as Queuing Networks (QNs), Layered Queuing Networks (LQNs) or stochastic process models. For these solution formalisms, a series of solution approaches exist, ranging from simple approximations, analytical approaches to full-scale simulations. This ensures that for each use case a suitable solution approach is available without having to maintain multiple models of the system. However, manual selection requires expert knowledge and manual intervention. Therefore, we aim to design an intelligent selection algorithm. This algorithm selects a suitable solution approach for every performance query. We define a performance query as a collection of requested performance metrics, a lower limit for the prediction accuracy and an upper limit for the time-to-result. In order to select a suitable solution approach for a query we first filter out all approaches that are not capable of predicting the requested performance metrics. For example, many analytical approaches are not capable of predicting response time distributions. In a next step, the selection algorithm needs to predict the accuracy and time-to-result for every solution approach. To predict the accuracy we plan to combine expert knowledge about the solvers with smart transformations, that can estimate how well the initial model is represented in the transformation output. For each simplification a transformation makes a number of penalty points, depending the level of simplification is awarded. For example, if a transformation represents loops in a probabilistic fashion only two penalty points per loop would be awarded, while a transformation that ignores passive resources would award a larger number of penalty points for each passive resource in the original model. Additional penalty points are awarded in case the applied solution approach calculates approximations instead of exact results. These penalty scores are determined once using expert knowledge. The resulting penalty score represents an indicator for the expected accuracy of a solution approach. We plan to utilize machine learning approaches to estimate the time-to-result. In general, estimating the time-to-result for performance predictions is challenging. However, we do not need to predict the time-to-result for a previously unseen model. After the inception of the performance model, each solution approach can be benchmarked to generate training data. From this training data, the machine learning approaches can then estimate the time-to-result for a specific request. The accuracy and time-to-result estimations will enable the design of an automated selection algorithm, that selects a suitable solution approach for each performance query. This allows for flexible performance predictions using a single model.}, author = {Eismann, Simon}, howpublished = {HotCloudperf 2018}, keywords = {Meta-models}, month = {04}, title = {{Flexible Performance Predictions at Run-time}}, year = 2018 }
• 
  TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research. J. von Kistowski; (2018, April).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kistowski-HotCloudPerf18, author = {von Kistowski, J{\'o}akim}, howpublished = {Talk at Workshop on Hot Topics in Cloud Computing Performance 2018}, keywords = {Power}, month = {04}, title = {{TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research}}, year = 2018 }
• 
  Measuring and Benchmarking Power Consumption and Energy Efficiency. J. von Kistowski; K.-D. Lange; J. A. Arnold; S. Sharma; J. Pais; H. Block; in Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 57–65.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{KiLaArShPaBl2018-ICPE-Power-Tutorial, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Lange, Klaus-Dieter and Arnold, Jeremy A. and Sharma, Sanjay and Pais, Johann and Block, Hansfried}, booktitle = {Companion of the 2018 ACM/SPEC International Conference on Performance Engineering}, keywords = {Power}, pages = {57–65}, publisher = {ACM}, series = {ICPE '18}, title = {{Measuring and Benchmarking Power Consumption and Energy Efficiency}}, year = 2018 }
• 
  Using Machine Learning for Recommending Service Demand Estimation Approaches. J. Grohmann; N. Herbst; S. Spinner; S. Kounev; in Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018) (2018). 473–480.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%.

  @inproceedings{GrHeSpKo-Closer18-RecServDemEst, abstract = {Service demands are key parameters in service and performance modeling. Hence, a variety of different approaches to service demand estimation exist in the literature. However, given a specific scenario, it is not trivial to select the currently best approach, since deep expertise in statistical estimation techniques is required and the requirements and characteristics of the application scenario might change over time (e.g., by varying load patterns). To tackle this problem, we propose the use of machine learning techniques to automatically recommend the best suitable approach for the target scenario. The approach works in an online fashion and can incorporate new measurement data and changing characteristics on-the-fly. Preliminary results show that executing only the recommended estimation approach achieves 99.6% accuracy compared to executing all approaches available, while speeding up the estimation time by 57%.}, author = {Grohmann, Johannes and Herbst, Nikolas and Spinner, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 8th International Conference on Cloud Computing and Services Science (CLOSER 2018)}, keywords = {Optimization}, month = {03}, organization = {INSTICC}, pages = {473–480}, publisher = {SciTePress}, title = {Using Machine Learning for Recommending Service Demand Estimation Approaches}, year = 2018 }
• 
  Model-Based Network Analysis and Optimization. S. Herrnleben; (2018, February).

  Invited Talk for Best Master Thesis Award 2018 of GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB).
  • [ BibTeX ]
  • [ Download ]
  @misc{BestMasterThesisMMB18, author = {Herrnleben, Stefan}, keywords = {DNI}, month = {02}, note = {<b>Invited Talk for Best Master Thesis Award 2018 of GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB).</b>}, title = {{Model-Based Network Analysis and Optimization}}, year = 2018 }
• 
  On the Value of Service Demand Estimation for Auto-Scaling. A. Bauer; J. Grohmann; N. Herbst; S. Kounev; in Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018) (2018). (Vol. 10740) 142–156.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.

  @inproceedings{BaGrHeKo2018-MMB-ServiceDemand, abstract = {In the context of performance models, service demands are key model parameters capturing the average time individual requests of different workload classes are actively processed. In a system under load, due to measurement interference, service demands normally cannot be measured directly, however, a number of estimation approaches exist based on high-level performance metrics. In this paper, we show that service demands provide significant benefits for implementing modern auto-scalers. Auto-scaling describes the process of dynamically adjusting the number of allocated virtual resources (e.g., virtual machines) in a data center according to the incoming workload. We demonstrate that even a simple auto-scaler that leverages information about service demands significantly outperforms auto-scalers solely based on CPU utilization measurements. This is shown by testing two approaches in three different scenarios. Our results show that the service demand-based auto-scaler outperforms the CPU utilization-based one in all scenarios. Our results encourage further research on the application of service demand estimates for resource management in data centers.}, address = {Cham}, author = {Bauer, Andr{\'e} and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of 19th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (MMB 2018)}, keywords = {Virtualization}, month = {02}, pages = {142–156}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{On the Value of Service Demand Estimation for Auto-Scaling}}, volume = 10740, year = 2018 }
• 
  SPEC CPU2017: Next-Generation Compute Benchmark. J. Bucek; K.-D. Lange; J. von Kistowski; in Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 41–42.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{BuLaKi2018-SPECCPU, address = {New York, NY, USA}, author = {Bucek, James and Lange, Klaus-Dieter and von Kistowski, J{\'o}akim}, booktitle = {{Companion of the 2018 ACM/SPEC International Conference on Performance Engineering}}, keywords = {t_poster}, pages = {41–42}, publisher = {ACM}, series = {ICPE '18}, title = {{SPEC CPU2017: Next-Generation Compute Benchmark}}, year = 2018 }
• 
  Tools for Declarative Performance Engineering. J. Walter; S. Eismann; J. Grohmann; D. Okanovic; S. Kounev; in Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 53–56.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.

  @inproceedings{WaEiGrOkKo2018-ICPE-Tools-for-DPE-Tutorial, abstract = {Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.}, address = {New York, NY, USA}, author = {Walter, J{\"u}rgen and Eismann, Simon and Grohmann, Johannes and Okanovic, Dusan and Kounev, Samuel}, booktitle = {Companion of the 2018 ACM/SPEC International Conference on Performance Engineering}, keywords = {DQL}, pages = {53–56}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '18}, title = {{Tools for Declarative Performance Engineering}}, year = 2018 }
• 
  A SPEC RG Cloud Group’s Vision on the Performance Challenges of FaaS Cloud Architectures. E. van Eyk; A. Iosup; C. L. Abad; J. Grohmann; S. Eismann; in Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 21–24.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  As a key part of the serverless computing paradigm, Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues. However, there are several performance-related issues surrounding the state-of-the-art FaaS platforms that can deter widespread adoption of FaaS, including sizeable overheads, unreliable performance, and new forms of the cost-performance trade-off. In this work we, the SPEC RG Cloud Group, identify six performance-related challenges that arise specifically in this FaaS model, and present our roadmap to tackle these problems in the near future. This paper aims at motivating the community to solve these challenges together.

  @inproceedings{vanEyk:2018:SRC:3185768.3186308, abstract = {As a key part of the serverless computing paradigm, Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues. However, there are several performance-related issues surrounding the state-of-the-art FaaS platforms that can deter widespread adoption of FaaS, including sizeable overheads, unreliable performance, and new forms of the cost-performance trade-off. In this work we, the SPEC RG Cloud Group, identify six performance-related challenges that arise specifically in this FaaS model, and present our roadmap to tackle these problems in the near future. This paper aims at motivating the community to solve these challenges together.}, address = {New York, NY, USA}, author = {van Eyk, Erwin and Iosup, Alexandru and Abad, Cristina L. and Grohmann, Johannes and Eismann, Simon}, booktitle = {Companion of the 2018 ACM/SPEC International Conference on Performance Engineering}, keywords = {SPEC}, pages = {21–24}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '18}, title = {A {SPEC RG} Cloud Group's Vision on the Performance Challenges of FaaS Cloud Architectures}, year = 2018 }
• 
  The Vision of Self-aware Reordering of Security Network Function Chains. L. Iffländer; J. Walter; S. Eismann; S. Kounev; in Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 1–4.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.

  @inproceedings{IfWaEiKo2018-ICPE-SSFC-Vision, abstract = {Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.}, address = {New York, NY, USA}, author = {Iffländer, Lukas and Walter, Jürgen and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering}, keywords = {Self-adaptive-systems}, pages = {1–4}, publisher = {ACM}, series = {ICPE '18}, title = {The Vision of Self-aware Reordering of Security Network Function Chains}, year = 2018 }
• 
  Energy Benchmarking. K.-D. Lange; J. von Kistowski; in Encyclopedia of Big Data Technologies, S. Sakr, A. Zomaya (eds.) (2018). 1–6.
  • [ BibTeX ]
  • [ URL ]
  @incollection{LaKi2018-BigDataEnergy, address = {Cham}, author = {Lange, Klaus-Dieter and von Kistowski, J{\'o}akim}, booktitle = {Encyclopedia of Big Data Technologies}, editor = {Sakr, Sherif and Zomaya, Albert}, keywords = {Power}, pages = {1–6}, publisher = {Springer International Publishing}, title = {Energy Benchmarking}, year = 2018 }
• 
  Predictive Maintenance for Industry 4.0. M. Züfle; (2018, April).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Zuefle-SelPhyS, author = {Z{\"u}fle, Marwin}, howpublished = {Talk at Workshop on Self-Awareness in Cyber-Physical Systems 2018}, keywords = {Talks}, month = {04}, title = {{Predictive Maintenance for Industry 4.0}}, year = 2018 }
• 
  SPEC Research Group Newsletter, vol. 2 no. 3. S. Kounev; A. van Hoorn; N. Herbst; A. Bauer; (2018, April).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-7, author = {Kounev, Samuel and van Hoorn, Andre and Herbst, Nikolas and Bauer, Andr{\'e}}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 2 no. 3}}, year = 2018 }
• 
  Modern time-series forecasting as enabler of proactive systems management. S. Kounev; (2018, June).
  • [ BibTeX ]
  @misc{Talk-Ko2018-VU_Amsterdam-Forecasting, address = {Amsterdam, The Netherlands}, author = {Kounev, Samuel}, howpublished = {Invited Talk, Seminar on Modern Distributed Systems, Vrije Universiteit Amsterdam, June 12, 2018}, keywords = {descartes}, month = {06}, title = {{Modern time-series forecasting as enabler of proactive systems management}}, year = 2018 }
• 
  Towards Self-Aware Industry 4.0 using a Layered Meta-Model. V. Lesch; (2018, April).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Lesch-SelPhyS, author = {Lesch, Veronika}, howpublished = {Talk at Workshop on Self-Awareness in Cyber-Physical Systems 2018}, keywords = {Optimization}, month = {04}, title = {{Towards Self-Aware Industry 4.0 using a Layered Meta-Model}}, year = 2018 }
• 
  Run-time Prediction of Power Consumption for Component Deployments. J. von Kistowski; M. Deffner; S. Kounev; in Proceedings of the 15th IEEE International Conference on Autonomic Computing (ICAC 2018) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The Power consumption of servers in data centers depends greatly on the software running on each server and how it interacts with the hardware. Different deployments of distributed software components on heterogeneous servers can lead to significant differences in power consumption, depending on the server allocation and the current workload. As workloads and load intensity change, components may be re-deployed or exchanged in order to reduce the power consumption for the current load profile. The decision on which component to place on which server during run-time remains difficult as the power consumption that would result from such a placement remains unknown. Existing work on component deployment optimization at run-time focuses on maximizing performance or considers power in the context of static design time decisions. In this paper, we introduce a model to predict the power consumption of component placements at run-time based on the load and power profile collected for a running distributed application in a heterogeneous environment. In addition, we present a model that enables the use of our approach without dedicated power measurement devices, predicting power consumption based on load intensity and performance counters. We show that we can predict the power consumption of two different distributed web applications with a mean absolute percentage error of 2.21% and with an error of 1.04% when predicting a previously unobserved load level.

  @inproceedings{KiDeKo2018-ICAC-PowerPrediction, abstract = {The Power consumption of servers in data centers depends greatly on the software running on each server and how it interacts with the hardware. Different deployments of distributed software components on heterogeneous servers can lead to significant differences in power consumption, depending on the server allocation and the current workload. As workloads and load intensity change, components may be re-deployed or exchanged in order to reduce the power consumption for the current load profile. The decision on which component to place on which server during run-time remains difficult as the power consumption that would result from such a placement remains unknown. Existing work on component deployment optimization at run-time focuses on maximizing performance or considers power in the context of static design time decisions. In this paper, we introduce a model to predict the power consumption of component placements at run-time based on the load and power profile collected for a running distributed application in a heterogeneous environment. In addition, we present a model that enables the use of our approach without dedicated power measurement devices, predicting power consumption based on load intensity and performance counters. We show that we can predict the power consumption of two different distributed web applications with a mean absolute percentage error of 2.21% and with an error of 1.04% when predicting a previously unobserved load level.}, author = {von Kistowski, J{\'o}akim and Deffner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 15th IEEE International Conference on Autonomic Computing (ICAC 2018)}, keywords = {Power}, month = {09}, title = {{Run-time Prediction of Power Consumption for Component Deployments}}, year = 2018 }
• 
  The Vision of Self-Aware Performance Models. J. Grohmann; S. Eismann; S. Kounev; in 2018 IEEE International Conference on Software Architecture Companion (ICSA-C) (2018). 60–63.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Performance models are necessary components of self-aware computing systems, as they allow such systems to reason about their own state and behavior. Research in this field has developed a multitude of approaches to create, maintain, and solve performance models. In this paper, we propose a meta-self-aware computing approach making the processes of model creation, maintenance and solution themselves self-aware. This enables the automated selection and adaption of software performance engineering approaches specifically tailored to the system under study.

  @inproceedings{GrEiKo-ICSA18-Vision, abstract = {Performance models are necessary components of self-aware computing systems, as they allow such systems to reason about their own state and behavior. Research in this field has developed a multitude of approaches to create, maintain, and solve performance models. In this paper, we propose a meta-self-aware computing approach making the processes of model creation, maintenance and solution themselves self-aware. This enables the automated selection and adaption of software performance engineering approaches specifically tailored to the system under study.}, author = {Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, booktitle = {2018 IEEE International Conference on Software Architecture Companion (ICSA-C)}, keywords = {Self-adaptive-systems}, month = {04}, pages = {60–63}, title = {{The Vision of Self-Aware Performance Models}}, year = 2018 }
• 
  SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network. A. Greubel; A. Dmitrienko; S. Kounev; in Proceedings of the 34th Annual Computer Security Applications Conference (2018).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{GrDmKo2018-ACSAC-SmarTor, address = {New York, NY, USA}, author = {Greubel, Andre and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {Proceedings of the 34th Annual Computer Security Applications Conference}, keywords = {sss-group}, month = 12, organization = {ACM}, series = {ACSAC '18}, title = {{SmarTor: Smarter Tor with Smart Contracts: Improving resilience of topology distribution in the Tor network}}, year = 2018 }
• 
  Efficient and Effective Ransomware Detection in Databases. C. Hagen; A. Dmitrienko; L. Iffländer; M. Jobst; S. Kounev; in 34th Annual Computer Security Applications Conference (ACSAC) (2018).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{Hagen2018_EfficientandEffectiveRansomwareDetectioninDatabases, author = {Hagen, Christoph and Dmitrienko, Alexandra and Iffländer, Lukas and Jobst, Michael and Kounev, Samuel}, booktitle = {34th Annual Computer Security Applications Conference (ACSAC)}, keywords = {t_poster}, month = 12, organization = {ACM}, title = {Efficient and Effective Ransomware Detection in Databases}, year = 2018 }
• 
  TeaStore: A Micro-Service Reference Application for Cloud Researchers. S. Eismann; J. v. Kistowski; J. Grohmann; A. Bauer; N. Schmitt; N. Herbst; S. Kounev; in Proceedings of 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion) (2018). 11–12.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed cloud applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. Further, most cloud reference applications are difficult to deploy and run. In this paper, we present the TeaStore, a micro-service-based test and reference cloud application. TeaStore offers services with various performance characteristics and a high degree of freedom regarding its deployment and configuration to be used as a cloud reference application for researchers. The TeaStore is designed for the evaluation of performance modeling and resource management techniques. We invite cloud researchers to use the TeaStore and provide it open-source, extendable, easily deployable and monitorable.

  @inproceedings{EiKiGrBaScHeKo2018-UCC-TeastorePoster, abstract = {Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed cloud applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. Further, most cloud reference applications are difficult to deploy and run. In this paper, we present the TeaStore, a micro-service-based test and reference cloud application. TeaStore offers services with various performance characteristics and a high degree of freedom regarding its deployment and configuration to be used as a cloud reference application for researchers. The TeaStore is designed for the evaluation of performance modeling and resource management techniques. We invite cloud researchers to use the TeaStore and provide it open-source, extendable, easily deployable and monitorable.}, author = {Eismann, Simon and v. Kistowski, J{\'o}akim and Grohmann, Johannes and Bauer, Andr{\'e} and Schmitt, Norbert and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion)}, keywords = {t_poster}, month = 12, pages = {11–12}, publisher = {IEEE}, title = {TeaStore: A Micro-Service Reference Application for Cloud Researchers}, year = 2018 }
• 
  TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research. S. Eismann; (2018, November).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Modern distributed component and microservice applications can be deployed in various ways and configured using different settings and software stacks. These applications have complex performance characteristics, as the constituent services feature different bottlenecks that may even change over time, depending on the usage profile. In general, microservice architectures present new challenges in the areas of performance monitoring, performance evaluation, performance modeling and performance prediction. The performance community has proposed many approaches that utilize the degrees of freedom of such applications at different points of the software life-cycle to tackle these emerging challenges. Verifying, comparing, and evaluating the results of such research is difficult. To enable practical evaluation, researchers need a software application that they can deploy as reference, offers realistic degrees of freedom and sufficiently complex performance behavior. The software in question should be open source, provide sufficient instrumentation, and should produce results that enable analysis and comparison of research findings, all while being indicative of how the evaluated research would affect applications in production use. Real world distributed software is usually proprietary and cannot be used for experimentation. In addition, results from evaluations conducted using such software are difficult to reproduce and compare, as the software used remains inaccessible for other researchers. Existing and broadly used test software does not offer the necessary degrees of freedom and is often manually adapted. Some of the most widely used test and reference applications, such as RUBiS, Dell DVD Store or SPECjEnterprise2010 are outdated and not representative of modern real-world applications. Reference applications from software vendors such as the Sock Shop, on the other hand, use a state-of-the-art software stack that is representative of modern real-world applications. However, these applications do not contain complex business logic and therefore do not offer representative performance behavior. We present the TeaStore, a micro-services-based test and reference application that can be used as a benchmarking framework by researchers. The TeaStore consists of five services, each featuring unique performance characteristics and bottlenecks: • The WebUI service displays static HTML pages, which are enriched using calls to the REST APIs of the remaining services. • The Auth service handles session validation (SHA512) and password hashing (BCrypt). • A list of recommended products is supplied by the Recommender service based on the user’s previous purchases. • The product images are retrieved from the ImageProvider service, which either loads them from cache or from the HDD. • Access to the database is encapsulated by the Persistence service, which can be used with a variety of databases. Service discovery is implemented using the Netflix Ribbon client-side load balancer and a simplified implementation of the Netflix Eureka registry. This enables distributed deployment and high scalability. The TeaStore offers multiple deployment options, manual deployment of WAR files, public docker containers or deployment using container orchestration frameworks, such as Kubernetes. Deployment in a container orchestration framework enables dynamic autoscaling of service instances, container health monitoring and failure recovery. We provide docker containers with a tailored Kieker instrumentation and a central trace repository, which collects the monitoring traces from all service instances using a RabbitMQ Server. Using these monitoring traces, tools such as the Performance Model Extractor (PMX) [1] can be used to extract PCM models and evaluate the accuracy of these models. As the TeaStore is continuously developed, it can be used to evaluate approaches that attempt to incorporate architectural performance models into the DevOps cycle, such as [2]. In general, the services’ different resource usage profiles enable performance and efficiency optimization with non-trivial service placement and resource provisioning decisions. [1] Jürgen Walter, Christian Stier, Heiko Koziolek, and Samuel Kounev. An Expandable Extraction Framework for Architectural Performance Models. InProceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17), L'Aquila, Italy, April 2017. ACM. April 2017 [2] Manar Mazkatli and Anne Koziolek. 2018. Continuous Integration of Performance Model. In Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (ICPE '18). ACM, New York, NY, USA, 153-158. DOI: https://doi.org/10.1145/3185768.3186285

  @misc{talk-Eismann-SSP2018, abstract = {Modern distributed component and microservice applications can be deployed in various ways and configured using different settings and software stacks. These applications have complex performance characteristics, as the constituent services feature different bottlenecks that may even change over time, depending on the usage profile. In general, microservice architectures present new challenges in the areas of performance monitoring, performance evaluation, performance modeling and performance prediction. The performance community has proposed many approaches that utilize the degrees of freedom of such applications at different points of the software life-cycle to tackle these emerging challenges. Verifying, comparing, and evaluating the results of such research is difficult. To enable practical evaluation, researchers need a software application that they can deploy as reference, offers realistic degrees of freedom and sufficiently complex performance behavior. The software in question should be open source, provide sufficient instrumentation, and should produce results that enable analysis and comparison of research findings, all while being indicative of how the evaluated research would affect applications in production use. Real world distributed software is usually proprietary and cannot be used for experimentation. In addition, results from evaluations conducted using such software are difficult to reproduce and compare, as the software used remains inaccessible for other researchers. Existing and broadly used test software does not offer the necessary degrees of freedom and is often manually adapted. Some of the most widely used test and reference applications, such as RUBiS, Dell DVD Store or SPECjEnterprise2010 are outdated and not representative of modern real-world applications. Reference applications from software vendors such as the Sock Shop, on the other hand, use a state-of-the-art software stack that is representative of modern real-world applications. However, these applications do not contain complex business logic and therefore do not offer representative performance behavior. We present the TeaStore, a micro-services-based test and reference application that can be used as a benchmarking framework by researchers. The TeaStore consists of five services, each featuring unique performance characteristics and bottlenecks: • The WebUI service displays static HTML pages, which are enriched using calls to the REST APIs of the remaining services. • The Auth service handles session validation (SHA512) and password hashing (BCrypt). • A list of recommended products is supplied by the Recommender service based on the user’s previous purchases. • The product images are retrieved from the ImageProvider service, which either loads them from cache or from the HDD. • Access to the database is encapsulated by the Persistence service, which can be used with a variety of databases. Service discovery is implemented using the Netflix Ribbon client-side load balancer and a simplified implementation of the Netflix Eureka registry. This enables distributed deployment and high scalability. The TeaStore offers multiple deployment options, manual deployment of WAR files, public docker containers or deployment using container orchestration frameworks, such as Kubernetes. Deployment in a container orchestration framework enables dynamic autoscaling of service instances, container health monitoring and failure recovery. We provide docker containers with a tailored Kieker instrumentation and a central trace repository, which collects the monitoring traces from all service instances using a RabbitMQ Server. Using these monitoring traces, tools such as the Performance Model Extractor (PMX) [1] can be used to extract PCM models and evaluate the accuracy of these models. As the TeaStore is continuously developed, it can be used to evaluate approaches that attempt to incorporate architectural performance models into the DevOps cycle, such as [2]. In general, the services’ different resource usage profiles enable performance and efficiency optimization with non-trivial service placement and resource provisioning decisions. [1] Jürgen Walter, Christian Stier, Heiko Koziolek, and Samuel Kounev. An Expandable Extraction Framework for Architectural Performance Models. InProceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17), L'Aquila, Italy, April 2017. ACM. April 2017 [2] Manar Mazkatli and Anne Koziolek. 2018. Continuous Integration of Performance Model. In Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (ICPE '18). ACM, New York, NY, USA, 153-158. DOI: https://doi.org/10.1145/3185768.3186285}, author = {Eismann, Simon}, howpublished = {9th Symposium on Software Performance}, keywords = {Cloud}, month = 11, title = {{TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research}}, year = 2018 }
• 
  Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking. L. Iffländer; S. Geißler; J. Walter; L. Beierlieb; S. Kounev; in Proceedings of the 9th Symposium on Software Performance 2018 (SSP’18) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore’s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

  @inproceedings{Ifflaender2018_AddressingShortcomingsofExistingDDoSProtectionSoftwareUsingSoftware-DefinedNetworking, abstract = {Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore’s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.}, author = {Iffländer, Lukas and Geißler, Stefan and Walter, Jürgen and Beierlieb, Lukas and Kounev, Samuel}, booktitle = {Proceedings of the 9th Symposium on Software Performance 2018 (SSP'18)}, keywords = {t_workshop}, month = 11, title = {Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking}, year = 2018 }
• 
  Black-box Learning of Parametric Dependencies for Performance Models. V. Ackermann; J. Grohmann; S. Eismann; S. Kounev; in Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.

  @inproceedings{AcGrEiKo2018-MRT-DependencyModeling, abstract = {Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.}, author = {Ackermann, Vanessa and Grohmann, Johannes and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of 13th International Workshop on Models@run.time (MRT), co-located with ACM/IEEE 21st International Conference on Model Driven Engineering Languages and Systems (MODELS 2018)}, keywords = {Optimization}, month = 10, series = {CEUR Workshop Proceedings}, title = {Black-box Learning of Parametric Dependencies for Performance Models}, year = 2018 }
• 
  Towards Infrastructure-Aided Self-Organized Hybrid Platooning. C. Krupitzer; M. Segata; M. Breitbach; S. S. El-Tawab; S. Tomforde; C. Becker; in Proceedings of the IEEE Global Conference on Internet of Things (GCIoT 2018) (2018).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Nowadays, the technical feasibility of autonomous driving is out of the question. This technology opens the door for several research topics (e.g., information exchange, security of information) in the IoT domain. One such application is platooning: coordinated driving of vehicles in convoys. Therefore, such platoons set up Internets-of-Vehicles. Although platooning has been mainly researched on highways, the idea of platooning in smart cities can have several benefits, such as efficient use of roads, time-saving through route optimization, and minimizing traffic in peak times. In this paper, we present a traffic management approach for combining platooning on highways and urban areas. We explain the idea of platooning, how to manage it efficiently, and the differences between platooning on the freeway and in smart cities. Finally, we explore the exchange of information in these systems and highlight on information security.

  @inproceedings{KrSeBrElToBe2018-GCIoT-infrastructureaidedPlatooning, abstract = {Nowadays, the technical feasibility of autonomous driving is out of the question. This technology opens the door for several research topics (e.g., information exchange, security of information) in the IoT domain. One such application is platooning: coordinated driving of vehicles in convoys. Therefore, such platoons set up Internets-of-Vehicles. Although platooning has been mainly researched on highways, the idea of platooning in smart cities can have several benefits, such as efficient use of roads, time-saving through route optimization, and minimizing traffic in peak times. In this paper, we present a traffic management approach for combining platooning on highways and urban areas. We explain the idea of platooning, how to manage it efficiently, and the differences between platooning on the freeway and in smart cities. Finally, we explore the exchange of information in these systems and highlight on information security.}, address = {Alexandria, Egypt}, author = {Krupitzer, Christian and Segata, Michele and Breitbach, Martin and El-Tawab, Samy S. and Tomforde, Sven and Becker, Christian}, booktitle = {Proceedings of the IEEE Global Conference on Internet of Things (GCIoT 2018)}, keywords = {Intelligent_Transportation_Systems}, month = 12, publisher = {IEEE}, title = {{Towards Infrastructure-Aided Self-Organized Hybrid Platooning}}, year = 2018 }
• 
  TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research. J. von Kistowski; S. Eismann; N. Schmitt; A. Bauer; J. Grohmann; S. Kounev; in Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (2018). 223–236.

  Acceptance Rate: 29.5% (23/78)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.

  @inproceedings{KiEiScBaGrKo2018-MASCOTS-TeaStore, abstract = {Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.}, author = {von Kistowski, J{\'o}akim and Eismann, Simon and Schmitt, Norbert and Bauer, Andr{\'e} and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 26th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems}, keywords = {Power}, month = {09}, note = {{Acceptance Rate: 29.5\% (23/78)}}, pages = {223–236}, publisher = {IEEE Computer Society}, series = {MASCOTS '18}, title = {{TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research}}, year = 2018 }
• 
  Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges. N. Herbst; A. Bauer; S. Kounev; G. Oikonomou; E. van Eyk; G. Kousiouris; A. Evangelinou; R. Krebs; T. Brecht; C. L. Abad; A. Iosup; in ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS) (2018). 3(4) 19:1–19:36.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{HeBaKoOiEyKoEvKrBrAbIo2018-TOMPECS-CloudMetrics, address = {New York, NY, USA}, author = {Herbst, Nikolas and Bauer, Andr{\'e} and Kounev, Samuel and Oikonomou, Giorgos and van Eyk, Erwin and Kousiouris, George and Evangelinou, Athanasia and Krebs, Rouven and Brecht, Tim and Abad, Cristina L. and Iosup, Alexandru}, journal = {ACM Transactions on Modeling and Performance Evaluation of Computing Systems (ToMPECS)}, keywords = {SPEC}, month = {08}, number = 4, pages = {19:1–19:36}, publisher = {ACM}, title = {{Quantifying Cloud Performance and Dependability: Taxonomy, Metric Design, and Emerging Challenges}}, volume = 3, year = 2018 }
• 
  Methods and Benchmarks for Auto-Scaling Mechanisms in Elastic Cloud Environments. N. Herbst; Thesis; University of Würzburg, Germany. (2018, July).

  Distinction, SPEC Kaivalya Dixit Distinguished Dissertation Award 2018
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Herbst2018-phd, author = {Herbst, Nikolas}, keywords = {SPEC}, month = {07}, note = {<b>{Distinction, SPEC Kaivalya Dixit Distinguished Dissertation Award 2018}</b>}, school = {University of W{\"u}rzburg, Germany}, title = {Methods and Benchmarks for Auto-Scaling Mechanisms in Elastic Cloud Environments}, year = 2018 }
• 
  How we built a scalable micro-service application - The Tea Store. N. Herbst; (2018, July).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{talk-Herbst-ScrumScale, author = {Herbst, Nikolas}, howpublished = {Keynote Talk at the ScrumScale workshop, Oslo, Norway, July 5, 2018}, keywords = {SPEC}, month = {07}, title = {{How we built a scalable micro-service application - The Tea Store}}, year = 2018 }
• 
  CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts. A. Milenkoski; A. Iosup; S. Kounev; K. Sachs; D. E. Mularz; J. A. Curtiss; J. J. Ding; F. Rosenberg; P. Rygielski; in IEEE Cloud Computing (2018). 5(3) 65–76.
  • [ BibTeX ]
  @article{MiIoKoSaMuCuDiRoRy2018-IEEECC-cup, author = {Milenkoski, Aleksandar and Iosup, Alexandru and Kounev, Samuel and Sachs, Kai and Mularz, Diane E. and Curtiss, Jonathan A. and Ding, Jason J. and Rosenberg, Florian and Rygielski, Piotr}, journal = {IEEE Cloud Computing}, keywords = {Virtualization}, month = {06}, number = 3, pages = {65–76}, publisher = {IEEE}, title = {{CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts}}, volume = 5, year = 2018 }
• 
  Elasticity Measurement in CaaS Environments - Extending the Existing BUNGEE Elasticity Benchmark to AWS’s Elastic Container Service. N. Limbourg; Thesis; Dublin Institute of Technology; Kevin Street, Dublin 2, D08 X622, Ireland. (2018, June).
  • [ BibTeX ]
  @mastersthesis{Limbourg2018Elasticity, address = {Kevin Street, Dublin 2, D08 X622, Ireland}, author = {Limbourg, Nora}, keywords = {Thesis_supervised_by_Veronika_Lesch}, month = {06}, school = {Dublin Institute of Technology}, title = {{Elasticity Measurement in CaaS Environments - Extending the Existing BUNGEE Elasticity Benchmark to AWS's Elastic Container Service}}, type = {{Master Thesis}}, year = 2018 }
• 
  Design of a Shared Parking System with Special Attention to Security Aspects. S. Englert; Thesis; University of Würzburg. (2018, August).
  • [ Abstract ]
  • [ BibTeX ]
  As major cities are growing more and more, the parking situation in those cities is becoming more precarious than ever. Parking demand already exceeds the limited amount of parking spaces available, and thus people searching for a place to park generate a significant part of those cities traffic. This leads to frustration with drivers, more traffic jams, unnecessary use of petrol and further air pollution. The creation of new parking spaces is often either difficult or very expansive, which means existing parking spots have to be used more efficiently. Solutions realizing that idea to try and solve those problems are for the most part called intelligent parking systems. This thesis is focused on Shared Parking, a special kind of intelligent parking system, which opens the possibility of sharing a parking place between different people. This work features the design and implementation of such a system with special attention given to security aspects.

  @mastersthesis{2018-BA-Simon-Engelert, abstract = {As major cities are growing more and more, the parking situation in those cities is becoming more precarious than ever. Parking demand already exceeds the limited amount of parking spaces available, and thus people searching for a place to park generate a significant part of those cities traffic. This leads to frustration with drivers, more traffic jams, unnecessary use of petrol and further air pollution. The creation of new parking spaces is often either difficult or very expansive, which means existing parking spots have to be used more efficiently. Solutions realizing that idea to try and solve those problems are for the most part called intelligent parking systems. This thesis is focused on Shared Parking, a special kind of intelligent parking system, which opens the possibility of sharing a parking place between different people. This work features the design and implementation of such a system with special attention given to security aspects.}, author = {Englert, Simon}, keywords = {sss-group}, month = {08}, school = {University of W{\"u}rzburg}, title = {Design of a Shared Parking System with Special Attention to Security Aspects}, type = {Bachelor Thesis}, year = 2018 }
• 
  Self-Aware Computing: From Vision to Reality. S. Kounev; (2018, May).
  • [ BibTeX ]
  @misc{Talk-Ko2018-AlumniNetworkUkraine-Self_Aware_Computing, address = {Kiew, Ukraine}, author = {Kounev, Samuel}, howpublished = {Konferenz des Alumni-Netzwerks Ukraine-Deutschland, Nationale Taras-Schewtschenko-Universität Kiew, May 29, 2018, Kyiv, Ukraine}, keywords = {descartes}, month = {05}, title = {{Self-Aware Computing: From Vision to Reality}}, year = 2018 }

• 
  Benchmarking Intrusion Detection Systems with Adaptive Provisioning of Virtualized Resources. A. Milenkoski; K. R. Jayaram; S. Kounev; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  With the increasing popularity of virtualization, deploying intrusion detection systems (IDSes) in virtualized environments, for example, in virtual machines as virtualized network functions, has become an emerging practice. Modern virtualized environments feature on demand provisioning of virtualized processing and memory resources to virtual machines, dynamically adapting its intensity in order to meet resource demands. Such a provisioning may have a significant impact on many properties of an IDS deployed in a virtual machine, for example, on its attack detection accuracy. However, conventional metrics for quantifying IDS attack detection accuracy do not capture this impact, which may lead to inaccurate assessments of the IDS’s accuracy at detecting attacks. In this chapter, we discuss in detail on the impact of on demand provisioning of virtualized resources on IDS attack detection accuracy. Further, we discuss on relevant issues related to the use of conventional metrics for quantifying IDS attack detection accuracy. Finally, we present a preliminary metric and measurement methodologies, which allow for the accurate assessment of IDS attack detection accuracy taking on-demand resource provisioning into account.

  @incollection{MiJaKo2017-self, abstract = {With the increasing popularity of virtualization, deploying intrusion detection systems (IDSes) in virtualized environments, for example, in virtual machines as virtualized network functions, has become an emerging practice. Modern virtualized environments feature on demand provisioning of virtualized processing and memory resources to virtual machines, dynamically adapting its intensity in order to meet resource demands. Such a provisioning may have a significant impact on many properties of an IDS deployed in a virtual machine, for example, on its attack detection accuracy. However, conventional metrics for quantifying IDS attack detection accuracy do not capture this impact, which may lead to inaccurate assessments of the IDS’s accuracy at detecting attacks. In this chapter, we discuss in detail on the impact of on demand provisioning of virtualized resources on IDS attack detection accuracy. Further, we discuss on relevant issues related to the use of conventional metrics for quantifying IDS attack detection accuracy. Finally, we present a preliminary metric and measurement methodologies, which allow for the accurate assessment of IDS attack detection accuracy taking on-demand resource provisioning into account.}, address = {{Berlin Heidelberg, Germany}}, author = {Milenkoski, Aleksandar and Jayaram, K. R. and Kounev, Samuel}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {t_bookchapter}, publisher = {{Springer Verlag}}, title = {{Benchmarking Intrusion Detection Systems with Adaptive Provisioning of Virtualized Resources}}, year = 2017 }
• 
  Run-time Models for Online Performance and Resource Management in Data Centers. S. Spinner; A. Filieri; S. Kounev; M. Maggio; A. Robertsson; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ BibTeX ]
  • [ Download ]
  @incollection{SpFiKoMaRo2016-SelfawareComputing-RuntimeModels, address = {{Berlin Heidelberg, Germany}}, author = {Spinner, Simon and Filieri, Antonio and Kounev, Samuel and Maggio, Martina and Robertsson, Anders}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Run-time Models for Online Performance and Resource Management in Data Centers}}, year = 2017 }
• 
  Reference Scenarios for Self-Aware Computing. J. O. Kephart; M. Maggio; A. Diaconescu; H. Giese; H. Hoffmann; S. Kounev; A. Koziolek; P. Lewis; A. Robertsson; S. Spinner; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ BibTeX ]
  • [ Download ]
  @incollection{KeMaDiGiHoKoKoLeRoSp2017-SelfAwareComputing-ReferenceScenarios, address = {{Berlin Heidelberg, Germany}}, author = {Kephart, Jeffrey O. and Maggio, Martina and Diaconescu, Ada and Giese, Holger and Hoffmann, Henry and Kounev, Samuel and Koziolek, Anne and Lewis, Peter and Robertsson, Anders and Spinner, Simon}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Reference Scenarios for Self-Aware Computing}}, year = 2017 }
• 
  Predicting Power Consumption of High-Memory-Bandwidth Workloads. N. Schmitt; J. von Kistowski; S. Kounev; in Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering (2017). 353–356.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  High performance workloads with high bandwidth memory utilization are among the most power consuming software applications. When writing such applications, developers can directly influence power consumption of the final software through their choice of data size and traversal method, mostly due to caching characteristics. Explicit knowledge on how choices influence power consumption can thus lead to greater overall energy efficiency. In existing work, power prediction for memory accesses and high bandwidth applications requires either detailed measurement information on the system on which the software is executed or it is too generic, not taking significant aspects, such as caching and data size into account. In this paper, we propose a power model that bridges this gap by modeling power consumption based on concrete software properties, while considering hardware characteristics on a more abstract level, characterizing it primarily using publicly available data. The model is designed to enable developers to compare power consumption of implementation alternatives for high memory bandwidth software components. We validate our model by measuring modified versions of the high bandwidth benchmark stream. We show that our model can predict the relative change of power consumption due to implementation changes and the power consumption of a concrete system under test with an average error of 19 percent.

  @inproceedings{ScKiKo2017-ICPE-MemoryPower, abstract = {{High performance workloads with high bandwidth memory utilization are among the most power consuming software applications. When writing such applications, developers can directly influence power consumption of the final software through their choice of data size and traversal method, mostly due to caching characteristics. Explicit knowledge on how choices influence power consumption can thus lead to greater overall energy efficiency. In existing work, power prediction for memory accesses and high bandwidth applications requires either detailed measurement information on the system on which the software is executed or it is too generic, not taking significant aspects, such as caching and data size into account. In this paper, we propose a power model that bridges this gap by modeling power consumption based on concrete software properties, while considering hardware characteristics on a more abstract level, characterizing it primarily using publicly available data. The model is designed to enable developers to compare power consumption of implementation alternatives for high memory bandwidth software components. We validate our model by measuring modified versions of the high bandwidth benchmark stream. We show that our model can predict the relative change of power consumption due to implementation changes and the power consumption of a concrete system under test with an average error of 19 percent.}}, address = {New York, NY, USA}, author = {Schmitt, Norbert and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 8th ACM/SPEC on International Conference on Performance Engineering}, keywords = {Power}, pages = {353–356}, publisher = {ACM}, series = {ICPE '17}, title = {{Predicting Power Consumption of High-Memory-Bandwidth Workloads}}, year = 2017 }
• 
  Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language. S. Kounev; N. Huber; F. Brosig; S. Spinner; M. Baehr; in Jan Jürjens, Kurt Schneider (Hrsg.): Software Engineering 2017 (SE 2017), Fachtagung des GI-Fachbereichs Softwaretechnik, 21.-24. Februar 2017, Hannover, Germany (2017).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KoHuBrSpBa2017-SE-DML, address = {Hannover, Germany}, author = {Kounev, Samuel and Huber, Nikolaus and Brosig, Fabian and Spinner, Simon and Baehr, Manuel}, booktitle = {Jan J{\"u}rjens, Kurt Schneider (Hrsg.): Software Engineering 2017 (SE 2017), Fachtagung des GI-Fachbereichs Softwaretechnik, 21.-24. Februar 2017, Hannover, Germany}, keywords = {descartes}, month = {02}, publisher = {Gesellschaft f{\"u}r Informatik (GI)}, series = {Lecture Notes in Informatics (LNI)}, title = {{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}}, year = 2017 }
• 
  Self-Aware Performance and Resource Management in Shared IT Infrastructures. S. Kounev; (2017, January).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-HUAWEI-2017, author = {Kounev, Samuel}, howpublished = {Invited Talk, Huawei Headquarters, Shenzhen, China}, keywords = {descartes}, month = {01}, title = {{Self-Aware Performance and Resource Management in Shared IT Infrastructures}}, year = 2017 }
• 
  Modeling and Extracting Load Intensity Profiles. J. von Kistowski; N. Herbst; S. Kounev; H. Groenda; C. Stier; S. Lehrig; in ACM Transactions on Autonomous and Adaptive Systems (TAAS) (2017). 11(4) 23:1–23:28.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.

  @article{KiHeKo-TAAS17-ModelExtractLoadProfiles, abstract = {Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Autonomic controllers, e.g., an advanced auto-scaling mechanism in a cloud computing context, can benefit from an abstracted load model as knowledge to reconfigure on time and precisely. Existing workload characterization approaches have limited support to capture variations the inter-arrival times of incoming work units over time (i.e., a variable load profile). For example, industrial and scientific benchmarks support constant or stepwise increasing load, or inter-arrival times defined by statistical distributions or recorded traces. These options show shortcomings either in representative character of load variation patterns or in abstraction and flexibility of their format. In this article, we present the Descartes Load Intensity Model (DLIM) approach addressing these issues. DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance is a compact formal description of a load intensity trace. DLIM-based tools provide features for benchmarking, performance and recorded load intensity trace analysis. As manually obtaining and maintaining DLIM instances becomes time consuming, we contribute three automated extraction methods and devised metrics for comparison and method selection. We discuss how these features are used to enhance system management approaches for adaptations during run-time, and how they are integrated into simulation contexts and enable benchmarking of elastic or adaptive behavior. We show that automatically extracted DLIM instances exhibit an average modeling error of 15.2\% over ten different real-world traces that cover between two weeks and seven months. These results underline DLIM model expressiveness. In terms of accuracy and processing speed, our proposed extraction methods for the descriptive models are comparable to existing time series decomposition methods. Additionally, we illustrate DLIM applicability by outlining approaches of workload modeling in systems engineering that employ or rely on our proposed load intensity modeling formalism.}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas and Kounev, Samuel and Groenda, Henning and Stier, Christian and Lehrig, Sebastian}, journal = {ACM Transactions on Autonomous and Adaptive Systems (TAAS)}, keywords = {Self-adaptive-systems}, month = {01}, number = 4, pages = {23:1–23:28}, publisher = {ACM}, title = {{Modeling and Extracting Load Intensity Profiles}}, volume = 11, year = 2017 }
• 
  Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language. S. Kounev; (2017, February).
  • [ BibTeX ]
  • [ Download ]
  @misc{Talk-Ko2017-SE-DML, address = {Hannover, Germany}, author = {Kounev, Samuel}, howpublished = {Software Engineering 2017 (SE 2017), Fachtagung des GI-Fachbereichs Softwaretechnik, 21.-24. Feb 2017, Hannover, Germany}, keywords = {descartes}, month = {02}, title = {{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}}, year = 2017 }
• 
  Software Architectures for Self-Protection in IaaS Clouds. K. R. Jayaram; A. Milenkoski; S. Kounev; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this chapter, we focus on software architectures for self-protection in IaaS clouds. IaaS clouds, especially hybrid clouds, are becoming increasingly popular because of the need for developers and enterprises to dynamically increase/decrease their use of computing resources to adapt quickly to market forces and customer demands, reduce costs, and increase fault tolerance. However, the adoption of public IaaS and hybrid clouds by enterprises is slower than expected because the current hybrid cloud infrastructures do not provide scalable and efficient mechanisms to prevent software tampering and configuration errors and ensure the trustworthiness and integrity of the software stack executing a hybrid application workload; or to enforce governmental privacy and audit regulations by ensuring that remote data and computation do not cross specified geographic boundaries. We discuss the recent research on integrating intrusion detection systems in IaaS infrastructures, as well as hardware-rooted integrity verification and geographic fencing to address the concerns outlined above.

  @incollection{JaMiKo2017-self, abstract = {In this chapter, we focus on software architectures for self-protection in IaaS clouds. IaaS clouds, especially hybrid clouds, are becoming increasingly popular because of the need for developers and enterprises to dynamically increase/decrease their use of computing resources to adapt quickly to market forces and customer demands, reduce costs, and increase fault tolerance. However, the adoption of public IaaS and hybrid clouds by enterprises is slower than expected because the current hybrid cloud infrastructures do not provide scalable and efficient mechanisms to prevent software tampering and configuration errors and ensure the trustworthiness and integrity of the software stack executing a hybrid application workload; or to enforce governmental privacy and audit regulations by ensuring that remote data and computation do not cross specified geographic boundaries. We discuss the recent research on integrating intrusion detection systems in IaaS infrastructures, as well as hardware-rooted integrity verification and geographic fencing to address the concerns outlined above.}, address = {{Berlin Heidelberg, Germany}}, author = {Jayaram, K. R. and Milenkoski, Aleksandar and Kounev, Samuel}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {t_bookchapter}, publisher = {{Springer Verlag}}, title = {{Software Architectures for Self-Protection in IaaS Clouds}}, year = 2017 }
• 
  State of the Art in Architectures for Self-Aware Computing Systems. H. Giese; T. Vogel; A. Diaconescu; S. Götz; N. Bencomo; K. Geihs; S. Kounev; K. Bellman; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ BibTeX ]
  • [ Download ]
  @incollection{GiVoDiGoBeGeKoBe2016-SelfawareComputing-StateoftheArt, address = {{Berlin Heidelberg, Germany}}, author = {Giese, Holger and Vogel, Thomas and Diaconescu, Ada and G{\"o}tz, Sebastian and Bencomo, Nelly and Geihs, Kurt and Kounev, Samuel and Bellman, Kirstie}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{State of the Art in Architectures for Self-Aware Computing Systems}}, year = 2017 }
• 
  Architectural Concepts for Self-Aware Computing Systems. H. Giese; T. Vogel; A. Diaconescu; S. Götz; S. Kounev; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ BibTeX ]
  • [ Download ]
  @incollection{GiVoDiGoKo2016-SelfawareComputing-ArchitecturalConcepts, address = {{Berlin Heidelberg, Germany}}, author = {Giese, Holger and Vogel, Thomas and Diaconescu, Ada and G{\"o}tz, Sebastian and Kounev, Samuel}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Architectural Concepts for Self-Aware Computing Systems}}, year = 2017 }
• 
  Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language. N. Huber; F. Brosig; S. Spinner; S. Kounev; M. Bähr; in IEEE Transactions on Software Engineering (TSE) (2017). 43(5) 432–452.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.

  @article{HuBrSpKoBa2017-TSE-DML, abstract = {Modern IT systems have increasingly distributed and dynamic architectures providing flexibility to adapt to changes in the environment and thus enabling higher resource efficiency. However, these benefits come at the cost of higher system complexity and dynamics. Thus, engineering systems that manage their end-to-end application performance and resource efficiency in an autonomic manner is a challenge. In this article, we present a holistic model-based approach for self-aware performance and resource management leveraging the Descartes Modeling Language (DML), an architecture-level modeling language for online performance and resource management. We propose a novel online performance prediction process that dynamically tailors the model solving depending on the requirements regarding accuracy and overhead. Using these prediction capabilities, we implement a generic model-based control loop for proactive system adaptation. We evaluate our model-based approach in the context of two representative case studies showing that with the proposed methods, significant resource efficiency gains can be achieved while maintaining performance requirements. These results represent the first end-to-end validation of our approach, demonstrating its potential for self-aware performance and resource management in the context of modern IT systems and infrastructures.}, author = {Huber, Nikolaus and Brosig, Fabian and Spinner, Simon and Kounev, Samuel and B{\"a}hr, Manuel}, journal = {IEEE Transactions on Software Engineering (TSE)}, keywords = {DQL}, number = 5, pages = {432–452}, publisher = {IEEE Computer Society}, title = {{Model-Based Self-Aware Performance and Resource Management Using the Descartes Modeling Language}}, volume = 43, year = 2017 }
• 
  Self-Awareness of Cloud Applications. A. Iosup; X. Zhu; A. Merchant; E. Kalyvianaki; M. Maggio; S. Spinner; T. Abdelzaher; O. Mengshoel; S. Bouchenak; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ BibTeX ]
  • [ Download ]
  @incollection{IoZhMeKaMaSpAbMeBo2016-SelfawareComputing-CloudApplications, address = {{Berlin Heidelberg, Germany}}, author = {Iosup, Alexandru and Zhu, Xiaoyun and Merchant, Arif and Kalyvianaki, Eva and Maggio, Martina and Spinner, Simon and Abdelzaher, Tarek and Mengshoel, Ole and Bouchenak, Sara}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {t_bookchapter}, publisher = {{Springer Verlag}}, title = {{Self-Awareness of Cloud Applications}}, year = 2017 }
• 
  The Notion of Self-Aware Computing. S. Kounev; P. Lewis; K. Bellman; N. Bencomo; J. Camara; A. Diaconescu; L. Esterle; K. Geihs; H. Giese; S. Götz; P. Inverardi; J. Kephart; A. Zisman; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ BibTeX ]
  • [ Download ]
  @incollection{KoLeBeBeCaDiEsGeGiGoInKeZi2016-SelfawareComputing-TheNotion, address = {{Berlin Heidelberg, Germany}}, author = {Kounev, Samuel and Lewis, Peter and Bellman, Kirstie and Bencomo, Nelly and Camara, Javier and Diaconescu, Ada and Esterle, Lukas and Geihs, Kurt and Giese, Holger and G{\"o}tz, Sebastian and Inverardi, Paola and Kephart, Jeffrey and Zisman, Andrea}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{The Notion of Self-Aware Computing}}, year = 2017 }
• 
  Self-Aware Computing Systems S. Kounev; J. O. Kephart; A. Milenkoski; X. Zhu; (2017). Springer Verlag, Berlin Heidelberg, Germany.
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  @book{KoKeMiZh2017-self, address = {{Berlin Heidelberg, Germany}}, author = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Self-Aware Computing Systems}}, year = 2017 }
• 
  Der Grader PABS. L. Iffländer; A. Dallmann; in Automatische Bewertung in der Programmierausbildung, O. J. Bott, P. Fricke, U. Priss, M. Striewe (eds.) (2017). (Vol. 6) 241–254.
  • [ BibTeX ]
  @incollection{IfDaPabs, author = {Iffländer, Lukas and Dallmann, Alexander}, booktitle = {Automatische Bewertung in der Programmierausbildung}, chapter = 15, editor = {Bott, Oliver J. and Fricke, Peter and Priss, Uta and Striewe, Michael}, keywords = {t_bookchapter}, pages = {241-254}, publisher = {ELAN e.V. and Waxmann Verlag}, series = {Digitale Medien in der Hochschullehre}, title = {Der Grader PABS}, volume = 6, year = 2017 }
• 
  Metrics and Benchmarks for Self-Aware Computing Systems. N. Herbst; S. Becker; S. Kounev; H. Koziolek; M. Maggio; A. Milenkoski; E. Smirni; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this chapter, we propose a list of metrics grouped by the MAPE-K paradigm for quantifying properties of self-aware computing systems. This set of metrics can be seen as a starting point toward benchmarking and comparing self-aware computing systems on a level-playing field. We discuss state-of-the art approaches in the related fields of self-adaptation and self-protection to identify commonalities in metrics for self-aware computing. We illustrate the need for benchmarking self-aware computing systems with the help of an approach that uncovers real-time characteristics of operating systems. Gained insights of this approach can be seen as a way of enhancing self-awareness by a measurement methodology on an ongoing basis. At the end of this chapter, we address new challenges in reference workload definition for benchmarking self-aware computing systems, namely load intensity patterns and burstiness modeling.

  @incollection{HeBeKoKoMaMiSm2017-self, abstract = {{In this chapter, we propose a list of metrics grouped by the MAPE-K paradigm for quantifying properties of self-aware computing systems. This set of metrics can be seen as a starting point toward benchmarking and comparing self-aware computing systems on a level-playing field. We discuss state-of-the art approaches in the related fields of self-adaptation and self-protection to identify commonalities in metrics for self-aware computing. We illustrate the need for benchmarking self-aware computing systems with the help of an approach that uncovers real-time characteristics of operating systems. Gained insights of this approach can be seen as a way of enhancing self-awareness by a measurement methodology on an ongoing basis. At the end of this chapter, we address new challenges in reference workload definition for benchmarking self-aware computing systems, namely load intensity patterns and burstiness modeling.}}, address = {{Berlin Heidelberg, Germany}}, author = {Herbst, Nikolas and Becker, Steffen and Kounev, Samuel and Koziolek, Heiko and Maggio, Martina and Milenkoski, Aleksandar and Smirni, Evgenia}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Metrics and Benchmarks for Self-Aware Computing Systems}}, year = 2017 }
• 
  CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches. T. F. Düllmann; R. Heinrich; A. van Hoorn; T. Pitakrat; J. Walter; F. Willnecker; in Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.

  @inproceedings{DuHeHoPiWaWi2017-CASPA, abstract = {Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.}, author = {D{\"u}llmann, Thomas F. and Heinrich, Robert and van Hoorn, Andre and Pitakrat, Teerat and Walter, J{\"u}rgen and Willnecker, Felix}, booktitle = {Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017)}, keywords = {PMX}, publisher = {IEEE}, title = {CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches}, year = 2017 }
• 
  Online Learning of Run-time Models for Performance and Resource Management in Data Centers. J. Walter; A. D. Marco; S. Spinner; P. Inverardi; S. Kounev; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.

  @incollection{WaMaSpInKo-learning, abstract = {In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.}, address = {{Berlin Heidelberg, Germany}}, author = {Walter, J{\"u}rgen and Marco, Antinisca Di and Spinner, Simon and Inverardi, Paola and Kounev, Samuel}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Online Learning of Run-time Models for Performance and Resource Management in Data Centers}}, year = 2017 }
• 
  Self-Aware Computing Systems: Related Concepts and Research Areas. J. Cámara; K. Bellman; J. Kephart; M. Autili; N. Bencomo; A. Diaconescu; H. Giese; S. Götz; P. Inverardi; S. Kounev; M. Tivoli; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ BibTeX ]
  • [ Download ]
  @incollection{CaBeKeAuBeDiGiGoInKoTi2016-SelfawareComputing-Related, address = {{Berlin Heidelberg, Germany}}, author = {C{\'a}mara, Javier and Bellman, Kirstie and Kephart, Jeffrey and Autili, Marco and Bencomo, Nelly and Diaconescu, Ada and Giese, Holger and G{\"o}tz, Sebastian and Inverardi, Paola and Kounev, Samuel and Tivoli, Massimo}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Self-Aware Computing Systems: Related Concepts and Research Areas}}, year = 2017 }
• 
  Spacecraft Autonomous Reaction Capabilities, Control Approaches and Self-Aware Computing. K. Schilling; J. Walter; S. Kounev; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Space exploration missions require very challenging autonomous reaction capabilities, as spacecraft have to react appropriately to the partially unknown environment in time critical situations, where direct human interaction is often impossible due to significant signal propagation delays related to the huge distances. In particular, interplanetary exploration missions are especially demanding; therefore the missions CASSINI-HUYGEN (landing on the Saturnian moon) and ROSETTA (the accompanying and landing on a comet) will be emphasized. Today also distributed, networked, small satellites for telecommunication and Earth observation applications promise a paradigm shift compared to the traditional multi-functional big spacecraft and could benefit from self-aware computing capabilities.

  @incollection{ScWaKo-space, abstract = {Space exploration missions require very challenging autonomous reaction capabilities, as spacecraft have to react appropriately to the partially unknown environment in time critical situations, where direct human interaction is often impossible due to significant signal propagation delays related to the huge distances. In particular, interplanetary exploration missions are especially demanding; therefore the missions CASSINI-HUYGEN (landing on the Saturnian moon) and ROSETTA (the accompanying and landing on a comet) will be emphasized. Today also distributed, networked, small satellites for telecommunication and Earth observation applications promise a paradigm shift compared to the traditional multi-functional big spacecraft and could benefit from self-aware computing capabilities.}, address = {{Berlin Heidelberg, Germany}}, author = {Schilling, Klaus and Walter, J{\"u}rgen and Kounev, Samuel}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Spacecraft Autonomous Reaction Capabilities, Control Approaches and Self-Aware Computing}}, year = 2017 }
• 
  Flexible Modeling of Data Center Networks for Capacity Management. P. Rygielski; Thesis; University of Würzburg, Germany. (2017, March).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @phdthesis{Rygielski2017-phd, author = {Rygielski, Piotr}, keywords = {DNI}, month = {03}, school = {University of W{\"u}rzburg, Germany}, title = {Flexible Modeling of Data Center Networks for Capacity Management}, year = 2017 }
• 
  TableVisor 2.0: Towards Full-Featured, Scalable and Hardware-Independent Multi Table Processing. S. Geissler; S. Gebert; S. Herrnleben; T. Zinner; R. Bauer; M. Jarschel; in NetSoft 2017 (2017).

  Best Student Paper Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Modern Software Defined Networking (SDN) appli- cations rely on sophisticated packet processing. However, there is a mismatch between control plane requirements and data plane capabilities caused by increasing hardware heterogeneity. To overcome this challenge, we propose TableVisor, a proxy-layer for the OpenFlow control channel that enables the flexible and scalable abstraction of multiple physical devices into one emu- lated data plane switch that meets the requirements of the control plane application. TableVisor registers with the SDN controller as a single switch with use-case specific capabilities. It translates the instructions and rules from the control application towards the appropriate physical device where they are executed. In this paper, we present the updated architecture and functionality of TableVisor as well as first evaluation results based on testbed experiments.

  @inproceedings{info3-inproceedings-2017-4, abstract = {Modern Software Defined Networking (SDN) appli- cations rely on sophisticated packet processing. However, there is a mismatch between control plane requirements and data plane capabilities caused by increasing hardware heterogeneity. To overcome this challenge, we propose TableVisor, a proxy-layer for the OpenFlow control channel that enables the flexible and scalable abstraction of multiple physical devices into one emu- lated data plane switch that meets the requirements of the control plane application. TableVisor registers with the SDN controller as a single switch with use-case specific capabilities. It translates the instructions and rules from the control application towards the appropriate physical device where they are executed. In this paper, we present the updated architecture and functionality of TableVisor as well as first evaluation results based on testbed experiments.}, author = {Geissler, Stefan and Gebert, Steffen and Herrnleben, Stefan and Zinner, Thomas and Bauer, Robert and Jarschel, Michael}, booktitle = {NetSoft 2017}, keywords = {SDN}, month = {02}, note = {<b>Best Student Paper Award</b>}, title = {TableVisor 2.0: Towards Full-Featured, Scalable and Hardware-Independent Multi Table Processing}, year = 2017 }
• 
  Online Workload Forecasting. N. Herbst; A. Amin; A. Andrzejak; L. Grunske; S. Kounev; O. J. Mengshoel; P. Sundararajan; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, X. Zhu, A. Milenkoski (eds.) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  This chapter gives a summary of the state-of-the-art approaches from different research fields that can be applied to continuously forecast future developments of time series data streams. More specifically, the input time series data contains continuously monitored metrics that quantify the amount of incoming workload units to a self-aware system. It is the goal of this chapter to identify and present approaches for online workload forecasting that are required for a self-aware system to act proactively in terms of problem prevention and optimization inferred from likely changes in their usage. The research fields covered are machine learning and time series analysis. We describe explicit limitations and advantages for each forecasting method.

  @incollection{HeAmAnGrMeSuKo2016-self, abstract = {{This chapter gives a summary of the state-of-the-art approaches from different research fields that can be applied to continuously forecast future developments of time series data streams. More specifically, the input time series data contains continuously monitored metrics that quantify the amount of incoming workload units to a self-aware system. It is the goal of this chapter to identify and present approaches for online workload forecasting that are required for a self-aware system to act proactively in terms of problem prevention and optimization inferred from likely changes in their usage. The research fields covered are machine learning and time series analysis. We describe explicit limitations and advantages for each forecasting method.}}, address = {{Berlin Heidelberg, Germany}}, author = {Herbst, Nikolas and Amin, Ayman and Andrzejak, Artur and Grunske, Lars and Kounev, Samuel and Mengshoel, Ole J. and Sundararajan, Priya}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Zhu, Xiaoyun and Milenkoski, Aleksandar}, keywords = {Optimization}, publisher = {{Springer Verlag}}, title = {{Online Workload Forecasting}}, year = 2017 }
• 
  The SERT Metric and the Impact of Server Configuration J. von Kistowski; K.-D. Lange; J. A. Arnold; H. Block; G. Darnell; J. Beckett; M. Tricker; (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The Server Efficiency Rating Tool (SERT) is an established tool for measuring and analyzing the energy efficiency of servers. The SERT suite measuresserver efficiency using multiple workloads, which in turn consist ofsmallscale mini‐workloads called worklets. Using multiple worklets enables the SERT suite to holistically explore the behavior of many different systems and enables thorough analysis. However, multiple workloads also result in multiple energy efficiency scores.  This document introduces the single SERT metric (SERT Efficiency Score), which can be used to easily compare systems using a single number. This document explains how the SERT metric is calculated. It also illustrates how a system under test (SUT) configuration and changes to this configuration can impact the SERT Efficiency Score and demonstrates this using a running example.

  @techreport{KiLaArBlDaBeTr2017-SERT-metric, abstract = {{The Server Efficiency Rating Tool (SERT) is an established tool for measuring and analyzing the energy efficiency of servers. The SERT suite measuresserver efficiency using multiple workloads, which in turn consist ofsmallscale mini‐workloads called worklets. Using multiple worklets enables the SERT suite to holistically explore the behavior of many different systems and enables thorough analysis. However, multiple workloads also result in multiple energy efficiency scores.  This document introduces the single SERT metric (SERT Efficiency Score), which can be used to easily compare systems using a single number. This document explains how the SERT metric is calculated. It also illustrates how a system under test (SUT) configuration and changes to this configuration can impact the SERT Efficiency Score and demonstrates this using a running example.}}, address = {7001 Heritage Village Plaza, Suite 225, Gainesville, VA 20155, USA}, author = {von Kistowski, J{\'o}akim and Lange, Klaus-Dieter and Arnold, Jeremy A. and Block, Hansfried and Darnell, Greg and Beckett, John and Tricker, Mike}, institution = {Standard Performance Evaluation Corporation (SPEC)}, keywords = {Power}, month = {03}, title = {{The SERT Metric and the Impact of Server Configuration}}, year = 2017 }
• 
  Self-Tuning Resource Demand Estimation. J. Grohmann; N. Herbst; S. Spinner; S. Kounev; in Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017) (2017). 21–26.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ DOI ]
  • [ Download ]
  The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.

  @inproceedings{GrHeSpKo2017-ICAC-RDE, abstract = {The average time a resource needs to process incoming requests in a monitored workload mix is a key parameter of stochastic performance models. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments.Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature each coming with different strengths and run-time overheads. Most approaches offer parameters in order to customize the behavior of the estimator influencing the estimation quality and the required computation time. However, their configuration usually requires exhaustive testing, as default parameters normally do not provide optimal performance.In this paper, we propose a self-tuning approach based on discrete optimization that can be used to automatically tune the parameters of resource demand estimation methods, tailoring them to the specific application scenario and thus improving their accuracy. We apply and compare different techniques on a representative data set with varying load levels and number of workload classes. We show that our selected approach for parameter tuning can automatically improve the estimation quality of certain estimators by up to 25%.}, author = {Grohmann, Johannes and Herbst, Nikolas and Spinner, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017)}, keywords = {Self-adaptive-systems}, month = {07}, pages = {21–26}, title = {{Self-Tuning Resource Demand Estimation}}, year = 2017 }
• 
  Design and Evaluation of a Proactive, Application-Aware Auto-Scaler. A. Bauer; N. Herbst; S. Kounev; in Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) (2017).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{BaHeKo2017-ICPE-Auto-Scaler-Tutorial, author = {Bauer, Andr{\'e} and Herbst, Nikolas and Kounev, Samuel}, booktitle = {{Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}}, keywords = {LIMBO}, month = {04}, title = {{Design and Evaluation of a Proactive, Application-Aware Auto-Scaler}}, year = 2017 }
• 
  Automated and Adaptable Decision Support for Software Performance Engineering. J. Walter; A. van Hoorn; S. Kounev; in Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.

  @inproceedings{WalterHoKo2017-DecisionSupport, abstract = {Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.}, author = {Walter, J{\"u}rgen and van Hoorn, Andre and Kounev, Samuel}, booktitle = {Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools}, keywords = {DQL}, month = 12, title = {{Automated and Adaptable Decision Support for Software Performance Engineering}}, year = 2017 }
• 
  Performance Assessment of Cloud Migrations from Network and Application Point of View. L. Iffländer; F. Wamser; C. Metter; P. Tran-Gia; S. Kounev; in Proceedings of 9th EAI International Conference on Mobile Networks and Management (MONAMI 2018) (2017).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{IfWaMeTrKo-Monami-Migration-Performance, author = {Iffländer, Lukas and Wamser, Florian and Metter, Christopher and Tran-Gia, Phuoc and Kounev, Samuel}, booktitle = {{Proceedings of 9th EAI International Conference on Mobile Networks and Management (MONAMI 2018)}}, keywords = {Virtualization}, month = 12, title = {Performance Assessment of Cloud Migrations from Network and Application Point of View}, year = 2017 }
• 
  Automating the Build Pipeline for Docker Container. N. Reed; J. Walter; S. Kounev; (2017, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{ReWaKo2017-SPP-DockerPipeline, author = {Reed, Nikolai and Walter, J{\"u}rgen and Kounev, Samuel}, howpublished = {Technical Talk at Symposion on Software Performance (SSP) 2017}, keywords = {descartes}, month = 11, title = {{Automating the Build Pipeline for Docker Container}}, year = 2017 }
• 
  Performance and Security Influence of Augmenting IDS using SDN and NFV. L. Iffländer; J. Stoll; (2017, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-IfSt-SSP2017-IDS-SDN-NFV, author = {Iffländer, Lukas and Stoll, Jonathan}, howpublished = {Technical Talk at Symposion on Software Performance (SSP) 2017}, keywords = {SDN}, month = 11, title = {Performance and Security Influence of Augmenting IDS using SDN and NFV}, year = 2017 }
• 
  Telescope: A Hybrid Forecast Method for Univariate Time Series. M. Züfle; A. Bauer; N. Herbst; V. Curtef; S. Kounev; in Proceedings of the International work-conference on Time Series (ITISE 2017) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Forecasting is an important part of the decision-making process and used in many fields like business, economics, finance, science, and engineering. According to the No-Free-Lunch-Theorem from 1997, there is no general forecasting method, that performs best for all time series. Instead, expert knowledge is needed to decide which forecasting method to choose for a specific time series with its own characteristics. Since a trial and error approach is very inefficient and expert knowledge is useful but a time-consuming task that cannot be fully automated, we present a new hybrid multi-step-ahead forecasting approach based on time series decomposition. Initial evaluations show that this hybrid approach improves the forecast accuracy compared to six existing forecasting methods while maintaining a short runtime.

  @inproceedings{ZBBHCK2017-ITISE-Telescope, abstract = {Forecasting is an important part of the decision-making process and used in many fields like business, economics, finance, science, and engineering. According to the No-Free-Lunch-Theorem from 1997, there is no general forecasting method, that performs best for all time series. Instead, expert knowledge is needed to decide which forecasting method to choose for a specific time series with its own characteristics. Since a trial and error approach is very inefficient and expert knowledge is useful but a time-consuming task that cannot be fully automated, we present a new hybrid multi-step-ahead forecasting approach based on time series decomposition. Initial evaluations show that this hybrid approach improves the forecast accuracy compared to six existing forecasting methods while maintaining a short runtime.}, author = {Z{\"u}fle, Marwin and Bauer, Andr{\'e} and Herbst, Nikolas and Curtef, Valentin and Kounev, Samuel}, booktitle = {{Proceedings of the International work-conference on Time Series (ITISE 2017)}}, keywords = {Optimization}, month = {09}, title = {{Telescope: A Hybrid Forecast Method for Univariate Time Series}}, year = 2017 }
• 
  Emulating the Power Consumption Behavior of Server Workloads using CPU Performance Counters. N. Schmitt; J. von Kistowski; S. Kounev; in Proceedings of the 25th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems (2017).

  (Short Paper) Acceptance Rate: 26/84 = 31%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The accurate measurement of a server's power consumption when running realistic workloads enables characterization of its energy efficiency and helps to make better provisioning and workload placement decisions. Information on the energy efficiency of a server for a given target workload can greatly influence such decisions and thus the final energy efficiency of a cluster or data center. However, measuring energy efficiency and power consumption of server applications has become challenging as applications are often distributed or require work intensive configuration, setup, and specialized load drivers for reproducible testing. As a result, it may be not feasible to perform tests using the actual workload that is to be deployed. We introduce an approach to create small-scale workloads that emulate the power consumption-relevant behavior of an application by deliberately triggering specific power relevant performance counter events. These workloads can then be easily deployed on a target server for fast and efficient power characterization. We validate the proposed approach by approximating the power consumption behavior of different workloads at multiple load levels. We show that our approach is capable of producing small-scale workloads that reflect the power consumption behavior of their reference applications over multiple load levels with a minimum error of less than 1%.

  @inproceedings{ScKiKo2017-MASCOTS-Pet, abstract = {The accurate measurement of a server's power consumption when running realistic workloads enables characterization of its energy efficiency and helps to make better provisioning and workload placement decisions. Information on the energy efficiency of a server for a given target workload can greatly influence such decisions and thus the final energy efficiency of a cluster or data center. However, measuring energy efficiency and power consumption of server applications has become challenging as applications are often distributed or require work intensive configuration, setup, and specialized load drivers for reproducible testing. As a result, it may be not feasible to perform tests using the actual workload that is to be deployed. We introduce an approach to create small-scale workloads that emulate the power consumption-relevant behavior of an application by deliberately triggering specific power relevant performance counter events. These workloads can then be easily deployed on a target server for fast and efficient power characterization. We validate the proposed approach by approximating the power consumption behavior of different workloads at multiple load levels. We show that our approach is capable of producing small-scale workloads that reflect the power consumption behavior of their reference applications over multiple load levels with a minimum error of less than 1%.}, author = {Schmitt, Norbert and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 25th IEEE International Symposium on the Modelling, Analysis, and Simulation of Computer and Telecommunication Systems}, keywords = {Power}, month = {09}, note = {{(Short Paper) Acceptance Rate: 26/84 = 31\%}}, series = {MASCOTS '17}, title = {{Emulating the Power Consumption Behavior of Server Workloads using CPU Performance Counters}}, year = 2017 }
• 
  Energy Efficiency Benchmarking Framework for GPGPU. T. Wahl; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2017, August).
  • [ BibTeX ]
  @mastersthesis{Wahl2017, address = {Am Hubland, Informatikgeb{\"a}ude, 97074 W{\"u}rzburg, Germany}, author = {Wahl, Tobias}, keywords = {Power}, month = {08}, school = {University of W{\"u}rzburg}, title = {{Energy Efficiency Benchmarking Framework for GPGPU}}, type = {{BachelorThesis}}, year = 2017 }
• 
  Towards a Scalability and Energy Efficiency Benchmark for VNF. N. Schmitt; J. von Kistowski; S. Kounev; in Proceedings of the 9th TPC Technology Conference on Performance Engineering and Benchmarking (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Network Function Virtualization (NFV) is the transfer of network functions from dedicated devices to high-volume commodity servers. It opens opportunities for flexibility and energy savings. Concrete insights on the flexibility of specific NFV environments require measurement methodologies and benchmarks. However, current benchmarks are not measuring the ability of a virtual network function (VNF) to scale either horizontally or vertically. We therefore envision a new benchmark that measures a VNF's ability to scale while evaluating its energy efficiency at the same time. Such a benchmark would enable the selection of a suitable VNF for changing demands, deployed at an existing or new resource landscape, while minimizing energy costs.

  @inproceedings{ScKiKo2017-TPCTC-BenchmarkVision, abstract = {Network Function Virtualization (NFV) is the transfer of network functions from dedicated devices to high-volume commodity servers. It opens opportunities for flexibility and energy savings. Concrete insights on the flexibility of specific NFV environments require measurement methodologies and benchmarks. However, current benchmarks are not measuring the ability of a virtual network function (VNF) to scale either horizontally or vertically. We therefore envision a new benchmark that measures a VNF's ability to scale while evaluating its energy efficiency at the same time. Such a benchmark would enable the selection of a suitable VNF for changing demands, deployed at an existing or new resource landscape, while minimizing energy costs.}, author = {Schmitt, Norbert and von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 9th TPC Technology Conference on Performance Engineering and Benchmarking}, keywords = {Power}, month = {08}, series = {TPCTC '17}, title = {{Towards a Scalability and Energy Efficiency Benchmark for VNF}}, year = 2017 }
• 
  Stochastic Models for Self-Aware Computing in Data Centers. S. Kounev; (2017, July).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{talk-Kounev-ASMTA2017, author = {Kounev, Samuel}, howpublished = {Keynote Talk at the 24th International Conference on Analytical & Stochastic Modelling Techniques & Applications (ASMTA 2017), Newcastle-upon-Tyne, UK, July 10-12, 2017}, keywords = {Power}, month = {07}, title = {{Stochastic Models for Self-Aware Computing in Data Centers}}, year = 2017 }
• 
  Providing Model-Extraction-as-a-Service for Architectural Performance Models. J. Walter; S. Eismann; N. Reed; S. Kounev; in Proceedings of the 2017 Symposium on Software Performance (SSP) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.

  @inproceedings{WaEiReKo2017-SPP-MEaaS, abstract = {Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.}, author = {Walter, J{\"u}rgen and Eismann, Simon and Reed, Nikolai and Kounev, Samuel}, booktitle = {Proceedings of the 2017 Symposium on Software Performance (SSP)}, keywords = {t_workshop}, month = 11, title = {{Providing Model-Extraction-as-a-Service for Architectural Performance Models}}, year = 2017 }
• 
  Self-Aware Resource Management in Virtualized Data Centers. S. Spinner; Thesis; University of Würzburg, Germany. (2017, July).
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Spinner2017-phd, author = {Spinner, Simon}, keywords = {descartes}, month = {07}, school = {University of W{\"u}rzburg, Germany}, title = {Self-Aware Resource Management in Virtualized Data Centers}, year = 2017 }
• 
  Mapping of Service Level Objectives to Performance Queries. J. Walter; D. Okanovic; S. Kounev; in Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.

  @inproceedings{WaOkKo2017-WOSPC-SLO2Queries, abstract = {The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.}, author = {Walter, J{\"u}rgen and Okanovic, Dusan and Kounev, Samuel}, booktitle = {{Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}}, keywords = {DQL}, month = {04}, publisher = {ACM}, title = {{Mapping of Service Level Objectives to Performance Queries}}, year = 2017 }
• 
  Scalability Analysis of Cloud Software Services. G. Brataas; N. Herbst; S. Ivansek; J. Polutnik; in Companion Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017), Self Organizing Self Managing Clouds Workshop (SOSeMC 2017) (2017).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{BrHe2017CloudScalability, author = {Brataas, Gunnar and Herbst, Nikolas and Ivansek, Simon and Polutnik, Jure}, booktitle = {Companion Proceedings of the 14th IEEE International Conference on Autonomic Computing (ICAC 2017), Self Organizing Self Managing Clouds Workshop (SOSeMC 2017)}, keywords = {t_workshop}, month = {07}, publisher = {IEEE}, title = {{Scalability Analysis of Cloud Software Services}}, year = 2017 }
• 
  Autopilot: Enabling easy Benchmarking of Workload Energy Efficiency. J. von Kistowski; M. Deffner; J. A. Arnold; K.-D. Lange; J. Beckett; S. Kounev; in Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) (2017).

  Best Demo Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Benchmarking of energy efficiency is important as it helps researchers, customers, and developers to evaluate and compare the energy efficiency of software and hardware solutions. Developing and deploying energy-efficiency benchmarking workloads are challenging tasks, as work must be able to be executed in a power measurement environment using an energy-efficiency measurement methodology. The existing SPEC Chauffeur Worklet Development Kit (WDK) enables the development and use of custom workloads (called worklets) within a standardized power measurement methodology. However, it features no integration in development environments, making building and deployment of workloads challenging. We address this challenge by proposing Autopilot, a plugin for the Eclipse IDE. Autopilot enables fast and easy building and deployment of a workload under development on a system for testing. It also enables benchmark execution directly from the development environment.

  @inproceedings{KiDeArLaBeKo2017-ICPE-Autopilot, abstract = {Benchmarking of energy efficiency is important as it helps researchers, customers, and developers to evaluate and compare the energy efficiency of software and hardware solutions. Developing and deploying energy-efficiency benchmarking workloads are challenging tasks, as work must be able to be executed in a power measurement environment using an energy-efficiency measurement methodology. The existing SPEC Chauffeur Worklet Development Kit (WDK) enables the development and use of custom workloads (called worklets) within a standardized power measurement methodology. However, it features no integration in development environments, making building and deployment of workloads challenging. We address this challenge by proposing Autopilot, a plugin for the Eclipse IDE. Autopilot enables fast and easy building and deployment of a workload under development on a system for testing. It also enables benchmark execution directly from the development environment.}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Deffner, Maximilian and Arnold, Jeremy A. and Lange, Klaus-Dieter and Beckett, John and Kounev, Samuel}, booktitle = {Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}, keywords = {Power}, month = {04}, note = {<b>Best Demo Award</b>}, publisher = {ACM}, title = {{Autopilot: Enabling easy Benchmarking of Workload Energy Efficiency}}, year = 2017 }
• 
  An Experimental Performance Evaluation of Autoscaling Policies for Complex Workflows. A. Ilyushkin; A. Ali-Eldin; N. Herbst; A. V. Papadopoulos; B. Ghit; D. Epema; A. Iosup; in Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) (2017).

  Best Paper Candidate (1/4)
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Simplifying the task of resource management and scheduling for customers, while still delivering complex Quality-of-Service (QoS), is key to cloud computing. Many autoscaling policies have been proposed in the past decade to decide on behalf of cloud customers when and how to provision resources to a cloud application utilizing cloud elasticity features. However, in prior work, when a new policy is proposed, it is seldom compared to the state-of-the-art, and is often compared only to static provisioning using a predefined QoS target. This reduces the ability of cloud customers and of cloud operators to choose and deploy an autoscaling policy. In our work, we conduct an experimental performance evaluation of autoscaling policies, using as application model workflows, a commonly used formalism for automating resource management for applications with well-defined yet complex structure. We present a detailed comparative study of general, state-of-the-art, generic autoscaling policies, along with two new workflow-specific policies. To understand the performance differences between the 7 policies, we conduct various forms of pairwise and group comparisons. We report both individual and aggregated metrics. Our results highlights the trade-offs between the suggested policies, and thus enable a better understanding of the current state-of-the-art.

  @inproceedings{IlAlHePa-ICPE17-AutoScalerWorkflowEval, abstract = {Simplifying the task of resource management and scheduling for customers, while still delivering complex Quality-of-Service (QoS), is key to cloud computing. Many autoscaling policies have been proposed in the past decade to decide on behalf of cloud customers when and how to provision resources to a cloud application utilizing cloud elasticity features. However, in prior work, when a new policy is proposed, it is seldom compared to the state-of-the-art, and is often compared only to static provisioning using a predefined QoS target. This reduces the ability of cloud customers and of cloud operators to choose and deploy an autoscaling policy. In our work, we conduct an experimental performance evaluation of autoscaling policies, using as application model workflows, a commonly used formalism for automating resource management for applications with well-defined yet complex structure. We present a detailed comparative study of general, state-of-the-art, generic autoscaling policies, along with two new workflow-specific policies. To understand the performance differences between the 7 policies, we conduct various forms of pairwise and group comparisons. We report both individual and aggregated metrics. Our results highlights the trade-offs between the suggested policies, and thus enable a better understanding of the current state-of-the-art.}, address = {New York, NY, USA}, author = {Ilyushkin, Alexey and Ali-Eldin, Ahmed and Herbst, Nikolas and Papadopoulos, Alessandro V. and Ghit, Bogdan and Epema, Dick and Iosup, Alexandru}, booktitle = {Proceedings of the 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}, keywords = {Self-adaptive-systems}, month = {04}, note = {<b>Best Paper Candidate (1/4)</b>}, publisher = {ACM}, title = {{An Experimental Performance Evaluation of Autoscaling Policies for Complex Workflows}}, year = 2017 }
• 
  SPEC Research Group Newsletter, vol. 2 no. 2. S. Kounev; A. van Hoorn; P. Rygielski; N. Herbst; A. Bauer; (2017, April).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-6, author = {Kounev, Samuel and van Hoorn, Andre and Rygielski, Piotr and Herbst, Nikolas and Bauer, Andr{\'e}}, keywords = {se2}, month = {04}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 2 no. 2}}, year = 2017 }
• 
  An Expandable Extraction Framework for Architectural Performance Models. J. Walter; C. Stier; H. Koziolek; S. Kounev; in Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS’17) (2017).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.

  @inproceedings{WaStKoKo2017-QUDOS-PMXBuilder, abstract = {Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.}, author = {Walter, J{\"u}rgen and Stier, Christian and Koziolek, Heiko and Kounev, Samuel}, booktitle = {{Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17)}}, keywords = {t_workshop}, month = {04}, publisher = {ACM}, title = {{An Expandable Extraction Framework for Architectural Performance Models}}, year = 2017 }
• 
  SPECpower Research Working Group. J. von Kistowski; (2017, April).
  • [ BibTeX ]
  @misc{talk-Kistowski-SPECRG2017, author = {von Kistowski, J{\'o}akim}, howpublished = {Talk at SPEC RG Annual Meeting 2017}, keywords = {Power}, month = {04}, title = {{SPECpower Research Working Group}}, year = 2017 }
• 
  Local Approximation of Server Workloads. N. Schmitt; (2017, April).
  • [ BibTeX ]
  @misc{talk-Schmitt-SPECRG2017, author = {Schmitt, Norbert}, howpublished = {Talk at SPEC RG Annual Meeting 2017}, keywords = {Power}, month = {04}, title = {{Local Approximation of Server Workloads}}, year = 2017 }
• 
  Software Architectures for Self-Aware Computing Systems. S. Kounev; (2017, June).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{Talk-Ko2017-GI_SA-SA4SelfAwareComputing, address = {Leipzig, Germany}, author = {Kounev, Samuel}, howpublished = {Jahrestagung GI-Fachgruppe Architekturen 2017, 19.-20. Juni 2017, Leipzig, Germany}, keywords = {descartes}, month = {06}, title = {{Software Architectures for Self-Aware Computing Systems}}, year = 2017 }
• 
  The Self-Aware Data Center: From Vision to Reality. S. Kounev; (2017, July).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{Talk-Ko2017-TUD-SeAC_DataCenter, address = {Darmstadt, Germany}, author = {Kounev, Samuel}, howpublished = {Distinguished Lecture Series of the Collaborative Research Centre (DFG SFB) 1053 MAKI, July 6, 2017, Darmstadt, Germany}, keywords = {descartes}, month = {07}, title = {{The Self-Aware Data Center: From Vision to Reality}}, year = 2017 }

• 
  Variations in CPU Power Consumption. J. von Kistowski; H. Block; J. Beckett; C. Spradling; K.-D. Lange; S. Kounev; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Experimental analysis of computer systems' power consumption has become an integral part of system performance evaluation, efficiency management, and model-based analysis. As with all measurements, repeatability and reproducibility of power measurements is a major challenge. Nominally identical systems can have different power consumption running the same workload under otherwise identical conditions. This behavior can also be observed for individual system components. Specifically, CPU power consumption can vary amongst different samples of nominally identical CPUs. This in turn has a significant impact on the overall system power, considering that a system's processor is the largest and most dynamic power consumer of the overall system. The concrete impact of CPU sample power variations is unknown, as comprehensive studies about differences in power consumption for nominally identical systems are currently missing. We address this lack of studies by conducting measurements on four different processor types from two different architectures. For each of these types, we compare up to 30 physical processor samples with a total sum of 90 samples over all processor types. We analyze the variations in power consumption for the different samples using six different workloads over five load levels. Additionally, we analyze how these variations change for different processor core counts and architectures. The results of this paper show that selection of a processor sample can have a statistically significant impact on power consumption. With no correlation to performance, power consumption for nominally identical processors can differ as much as 29.6% in idle and 19.5% at full load. We also show that these variations change over different architectures and processor types.

  @inproceedings{KiBlBeSpLaKo2016-ICPE-PowerVariation, abstract = {{Experimental analysis of computer systems' power consumption has become an integral part of system performance evaluation, efficiency management, and model-based analysis. As with all measurements, repeatability and reproducibility of power measurements is a major challenge. Nominally identical systems can have different power consumption running the same workload under otherwise identical conditions. This behavior can also be observed for individual system components. Specifically, CPU power consumption can vary amongst different samples of nominally identical CPUs. This in turn has a significant impact on the overall system power, considering that a system's processor is the largest and most dynamic power consumer of the overall system. The concrete impact of CPU sample power variations is unknown, as comprehensive studies about differences in power consumption for nominally identical systems are currently missing. We address this lack of studies by conducting measurements on four different processor types from two different architectures. For each of these types, we compare up to 30 physical processor samples with a total sum of 90 samples over all processor types. We analyze the variations in power consumption for the different samples using six different workloads over five load levels. Additionally, we analyze how these variations change for different processor core counts and architectures. The results of this paper show that selection of a processor sample can have a statistically significant impact on power consumption. With no correlation to performance, power consumption for nominally identical processors can differ as much as 29.6\% in idle and 19.5\% at full load. We also show that these variations change over different architectures and processor types.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Block, Hansfried and Beckett, John and Spradling, Cloyce and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}, keywords = {Power}, month = {03}, publisher = {ACM}, title = {{Variations in CPU Power Consumption}}, year = 2016 }
• 
  Analysis of the Trade-offs in Different Modeling Approaches for Performance Prediction of Software Systems. S. Kounev; F. Brosig; P. Meier; S. Becker; A. Koziolek; H. Koziolek; P. Rygielski; in Software Engineering 2016 (SE 2016), Fachtagung des GI-Fachbereichs Softwaretechnik, 23.-26. Feb 2016, Vienna, Austria (2016). 47–48.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KoBrMeBeKoKoRy2016-SE-tradeoffs_perf_prediction_approaches, address = {Vienna, Austria}, author = {Kounev, Samuel and Brosig, Fabian and Meier, Philipp and Becker, Steffen and Koziolek, Anne and Koziolek, Heiko and Rygielski, Piotr}, booktitle = {Software Engineering 2016 (SE 2016), Fachtagung des GI-Fachbereichs Softwaretechnik, 23.-26. Feb 2016, Vienna, Austria}, keywords = {Formal_architecture_modeling}, month = {02}, pages = {47–48}, publisher = {GI}, series = {Lecture Notes in Informatics (LNI)}, title = {{Analysis of the Trade-offs in Different Modeling Approaches for Performance Prediction of Software Systems}}, year = 2016 }
• 
  A Reference Architecture for Online Performance Model Extraction in Virtualized Environments. S. Spinner; J. Walter; S. Kounev; in Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.

  @inproceedings{SpWaKo2015-WOSPC-ReferenceArchitecture, abstract = {Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.}, author = {Spinner, Simon and Walter, J{\"u}rgen and Kounev, Samuel}, booktitle = {{Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}}, keywords = {t_workshop}, month = {03}, title = {{A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}}, year = 2016 }
• 
  SPEC Research Group Newsletter, vol. 2 no. 1. S. Kounev; A. van Hoorn; P. Rygielski; (2016, March).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-5, author = {Kounev, Samuel and van Hoorn, Andre and Rygielski, Piotr}, keywords = {SPEC}, month = {03}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 2 no. 1}}, year = 2016 }
• 
  Analysis of the Trade-offs in Different Modeling Approaches for Performance Prediction of Software Systems. S. Kounev; (2016, February).
  • [ BibTeX ]
  • [ Download ]
  @misc{Talk-Ko2016-SE-tradeoffs_perf_prediction_approaches, address = {Vienna, Austria}, author = {Kounev, Samuel}, howpublished = {Software Engineering 2016 (SE 2016), Fachtagung des GI-Fachbereichs Softwaretechnik, 23.-26. Februar 2016, Vienna, Austria}, keywords = {descartes}, month = {02}, title = {{Analysis of the Trade-offs in Different Modeling Approaches for Performance Prediction of Software Systems}}, year = 2016 }
• 
  Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments. A. Busch; Q. Noorshams; S. Kounev; A. Koziolek; R. Reussner; E. Amrehn; in Software Engineering 2016 (SE 2016), Fachtagung des GI-Fachbereichs Softwaretechnik, 23.-26. März 2016, Vienna, Austria (2016). 27–28.
  • [ BibTeX ]
  @inproceedings{BuNoKoKoReAm-SE-IO_Workload_Characterization, address = {Vienna, Austria}, author = {Busch, Axel and Noorshams, Qais and Kounev, Samuel and Koziolek, Anne and Reussner, Ralf and Amrehn, Erich}, booktitle = {Software Engineering 2016 (SE 2016), Fachtagung des GI-Fachbereichs Softwaretechnik, 23.-26. M{\"a}rz 2016, Vienna, Austria}, keywords = {se2}, month = {02}, pages = {27–28}, publisher = {GI}, series = {Lecture Notes in Informatics (LNI)}, title = {{Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments}}, year = 2016 }
• 
  Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation. C. Müller; P. Rygielski; S. Spinner; S. Kounev; in Electronic Notes in Theoretical Computer Science (2016). 327 71–91.

  The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Abstract Due to the growing size of modern IT} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained LQN} models can benefit from three existing solvers: LINE, LQNS, LQSIM. LINE} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.

  @article{MuRySpKo2016-PASM-QPN-LQN-Transformation, abstract = {Abstract Due to the growing size of modern \{IT\} systems, their performance analysis becomes an even more challenging task. Existing simulators are unable to analyze the behavior of large systems in a reasonable time, whereas analytical methods suffer from the state space explosion problem. Fluid analysis techniques can be used to approximate the solution of high-order Markov chain models enabling time efficient analysis of large performance models. In this paper, we describe a model-to-model transformation from queueing Petri nets (QPN) into layered queueing networks (LQN). Obtained \{LQN\} models can benefit from three existing solvers: LINE, LQNS, LQSIM. \{LINE\} internally utilize fluid limits approximation to speed up the solving process for large models. We present the incentives for developing the automated model-to-model transformation and present a systematic approach that we followed in its design. We demonstrate the transformations using representative examples. Finally, we evaluate and compare the performance predictions of existing analytical, simulation and fluid analysis solvers. We analyze solvers' limitations, solving time, and memory consumption.}, author = {M{\"u}ller, Christoph and Rygielski, Piotr and Spinner, Simon and Kounev, Samuel}, journal = {Electronic Notes in Theoretical Computer Science}, keywords = {QPN}, note = {The 8th International Workshop on Practical Application of Stochastic Modeling, PASM 2016}, pages = {71–91}, publisher = {Elsevier}, title = {{Enabling Fluid Analysis for Queueing Petri Nets via Model Transformation}}, volume = 327, year = 2016 }
• 
  Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures. S. Kounev; (2016, January).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-InvFAU16, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), Department Informatik, Friedrich-Alexander-Universit{\"a}t Erlangen-N{\"u}rnberg, Erlangen, Germany}, keywords = {Talks}, month = {01}, title = {{Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures}}, year = 2016 }
• 
  Performance Assessment of Service Migration Strategies. L. Iffländer; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2016, January).
  • [ BibTeX ]
  • [ Download ]
  @mastersthesis{Ifflaender2016, address = {Am Hubland, Informatikgebäude, 97074 Würzburg, Germany}, author = {Iffländer, Lukas}, keywords = {Virtualization}, month = {01}, school = {University of Würzburg}, title = {Performance Assessment of Service Migration Strategies}, type = {Master Thesis}, year = 2016 }
• 
  DECLARE: Declarative Performance Engineering. A. van Hoorn; S. Kounev; D. Okanovi’c}}; J. Walter; (2016, January).
  • [ BibTeX ]
  • [ Download ]
  @misc{Talk-vanHoKo-SPP1593-2016, address = {Hannover, Germany}, author = {van Hoorn, Andr{\'e} and Kounev, Samuel and Okanovi{\'c}, Dusan and Walter, J{\"u}rgen}, howpublished = {Renewal Kickoff Workshop of the DFG Priority Programme 1593, Hannover, January 14-15, 2016}, keywords = {Talks}, month = {01}, title = {{DECLARE: Declarative Performance Engineering}}, year = 2016 }
• 
  Use Cases for Chauffeur. J. von Kistowski; (2016, January).
  • [ BibTeX ]
  @misc{talk-Kistowski-SPECSymposium2016, author = {von Kistowski, J{\'o}akim}, howpublished = {Invited Talk at SPEC Symposium North America 2016}, keywords = {Power}, month = {01}, title = {{Use Cases for Chauffeur}}, year = 2016 }
• 
  Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics. (SPEC-RG-2016-01), N. Herbst; R. Krebs; G. Oikonomou; G. Kousiouris; A. Evangelinou; A. Iosup; S. Kounev; (2016).
  • [ BibTeX ]
  • [ URL ]
  @techreport{SPEC-RG-2016-01-CloudMetrics, author = {Herbst, Nikolas and Krebs, Rouven and Oikonomou, Giorgos and Kousiouris, George and Evangelinou, Athanasia and Iosup, Alexandru and Kounev, Samuel}, institution = {SPEC Research Group — Cloud Working Group, Standard Performance Evaluation Corporation (SPEC)}, keywords = {Reliability}, number = {SPEC-RG-2016-01}, title = {{Ready for Rain? A View from SPEC Research on the Future of Cloud Metrics}}, year = 2016 }
• 
  Getting the Data. J. Happe; B. Klatt; M. Küster; F. Brosig; A. Wert; S. Spinner; C. Stier; H. Koziolek; in Modeling and Simulating Software Architectures - The Palladio Approach, R. H. Reussner, S. Becker, J. Happe, R. Heinrich, A. Koziolek, H. Koziolek, M. Kramer, K. Krogmann (eds.) (2016).

  In Press
  • [ BibTeX ]
  @incollection{HaKlKuBrWeSpStKo-2016-MITPress-Palladio, address = {Cambridge, MA}, author = {Happe, Jens and Klatt, Benjamin and K{\"u}ster, Martin and Brosig, Fabian and Wert, Alexander and Spinner, Simon and Stier, Christian and Koziolek, Heiko}, booktitle = {{Modeling and Simulating Software Architectures - The Palladio Approach}}, editor = {Reussner, Ralf H. and Becker, Steffen and Happe, Jens and Heinrich, Robert and Koziolek, Anne and Koziolek, Heiko and Kramer, Max and Krogmann, Klaus}, keywords = {t_bookchapter}, note = {In Press}, publisher = {MIT Press}, title = {{Getting the Data}}, year = 2016 }
• 
  Automated Parameterization of Performance Models from Measurements. G. Casale; S. Spinner; W. Wang; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Estimating parameters of performance models from empirical measurements is a critical task, which often has a major influence on the predictive accuracy of a model. This tutorial presents the problem of parameter estimation in queueing systems and queueing networks. The focus is on reliable estimation of the em arrival rates of the requests and of the em service demands they place at the servers. The tutorial covers common estimation techniques such as regression methods, maximum-likelihood estimation, and moment-matching, discussing their sensitivity with respect to data and model characteristics. The tutorial also demonstrates the automated estimation of model parameters using new open source tools.

  @inproceedings{CaSpWa2016-ICPE-AutomatedParameterizationTutorial, abstract = {Estimating parameters of performance models from empirical measurements is a critical task, which often has a major influence on the predictive accuracy of a model. This tutorial presents the problem of parameter estimation in queueing systems and queueing networks. The focus is on reliable estimation of the {\em arrival rates} of the requests and of the {\em service demands} they place at the servers. The tutorial covers common estimation techniques such as regression methods, maximum-likelihood estimation, and moment-matching, discussing their sensitivity with respect to data and model characteristics. The tutorial also demonstrates the automated estimation of model parameters using new open source tools.}, author = {Casale, Giuliano and Spinner, Simon and Wang, Weikun}, booktitle = {{Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}}, keywords = {LibReDE}, month = {03}, title = {{Automated Parameterization of Performance Models from Measurements}}, year = 2016 }
• 
  Automated Extraction of Network Traffic Models Suitable for Performance Simulation. P. Rygielski; V. Simko; F. Sittner; D. Aschenbrenner; S. Kounev; K. Schilling; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016). 27–35.

  Acceptance rate (Full Paper): 19/57 = 33%
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Data centers are increasingly becoming larger and dynamic due to virtualization. In order to leverage the performance modeling and prediction techniques, such as Palladio Component Model or Descartes Modeling Language, in such a dynamic environments, it is necessary to automate the model extraction. Building and maintaining such models manually is not feasible anymore due to their size and the level of details. This paper is focused on traffic models that are an essential part of network infrastructure. Our goal is to decompose real traffic dumps into models suitable for performance prediction using Descartes Network Infrastructure modeling approach. The main challenge was to efficiently encode an arbitrary signal in the form of simple traffic generators while maintaining the shape of the original signal. We show that a typical 15 minute long tcpdump trace can be compressed to 0.4--15% of its original size whereas the relative median of extraction error is 0% for the most of the 69 examined traces.

  @inproceedings{RySiSiAschKoSch2016-dni-traffic-extraction-icpe2016, abstract = {{Data centers are increasingly becoming larger and dynamic due to virtualization. In order to leverage the performance modeling and prediction techniques, such as Palladio Component Model or Descartes Modeling Language, in such a dynamic environments, it is necessary to automate the model extraction. Building and maintaining such models manually is not feasible anymore due to their size and the level of details. This paper is focused on traffic models that are an essential part of network infrastructure. Our goal is to decompose real traffic dumps into models suitable for performance prediction using Descartes Network Infrastructure modeling approach. The main challenge was to efficiently encode an arbitrary signal in the form of simple traffic generators while maintaining the shape of the original signal. We show that a typical 15 minute long tcpdump trace can be compressed to 0.4–15% of its original size whereas the relative median of extraction error is 0% for the most of the 69 examined traces.}}, author = {Rygielski, Piotr and Simko, Viliam and Sittner, Felix and Aschenbrenner, Doris and Kounev, Samuel and Schilling, Klaus}, booktitle = {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}, keywords = {DNI}, month = {03}, note = {{Acceptance rate (Full Paper): 19/57 = 33\%}}, pages = {27–35}, publisher = {ACM}, title = {{Automated Extraction of Network Traffic Models Suitable for Performance Simulation}}, year = 2016 }
• 
  Asking ``What?’’, Automating the ``How?’’: The Vision of Declarative Performance Engineering. J. Walter; A. van Hoorn; H. Koziolek; D. Okanovic; S. Kounev; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

  @inproceedings{WaHoKoOkKo2016-ICPE-DPE_Vision, abstract = {Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.}, author = {Walter, J{\"u}rgen and van Hoorn, Andre and Koziolek, Heiko and Okanovic, Dusan and Kounev, Samuel}, booktitle = {{Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}}, keywords = {DQL}, month = {03}, title = {{Asking ``What?'', Automating the ``How?'': The Vision of Declarative Performance Engineering}}, year = 2016 }
• 
  The Vision of Declarative Performance Engineering. J. Walter; A. van Hoorn; H. Koziolek; D. Okanovic; S. Kounev; in Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

  @inproceedings{WaHoKoOkKo2016-ICPE-DPE_Poster, abstract = {Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.}, author = {Walter, J{\"u}rgen and van Hoorn, Andre and Koziolek, Heiko and Okanovic, Dusan and Kounev, Samuel}, booktitle = {{Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}}, keywords = {DQL}, month = {03}, title = {{The Vision of Declarative Performance Engineering}}, year = 2016 }
• 
  Chameleon: Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism. A. Bauer; (2016, October).
  • [ BibTeX ]
  @misc{talk-Bauer-InvSpec16, author = {Bauer, Andr{\'e}}, howpublished = {Invited Talk, Spec Research - Cloud Working Group}, keywords = {Application-aware}, month = 10, title = {{Chameleon: Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism}}, year = 2016 }
• 
  Predicting Power Consumption in Virtualized Environments. J. von Kistowski; M. Schreck; S. Kounev; in Computer Performance Engineering: 13th European Workshop, EPEW 2016, Chios, Greece, October 5-7, 2016, Proceedings, D. Fiems, M. Paolieri, N. A. Platis (eds.) (2016). 79–93.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Energy efficiency and power consumption of data centers can be improved through conscientious placement of workloads on specific servers. Virtualization is commonly employed nowadays, as it allows for dynamic reallocation of work and abstraction from the concrete server hardware. The ability to predict the power consumption of workloads at different load levels is essential in this context. Prediction approaches can help to make better placement choices at run-time, as well as when purchasing new server equipment, by showing which servers are better suited for the execution of a given target workload. In existing work, power prediction for servers is limited to non-virtualized contexts or it does not take multiple load levels into account. Existing approaches also fail to leverage publicly available data on server efficiency and instead require experiments to be conducted on the target system. This makes these approaches unwieldy when making decision regarding systems that are not yet available to the decision maker. In this paper, we use the readily available data provided by the SPEC SERT to predict the power consumption of workloads for different load levels in virtualized environments. We evaluate our approach comparing predicted results against measurements of power consumption in multiple virtualized environment configurations on a target server that differs significantly from the reference system used for experimentation. We show that power consumption of CPU and storage loads can be reliably predicted with a prediction error of less than 15% across all tested virtualized environment configurations.

  @inproceedings{KiScKo-EPEW-VMPower, abstract = {Energy efficiency and power consumption of data centers can be improved through conscientious placement of workloads on specific servers. Virtualization is commonly employed nowadays, as it allows for dynamic reallocation of work and abstraction from the concrete server hardware. The ability to predict the power consumption of workloads at different load levels is essential in this context. Prediction approaches can help to make better placement choices at run-time, as well as when purchasing new server equipment, by showing which servers are better suited for the execution of a given target workload. In existing work, power prediction for servers is limited to non-virtualized contexts or it does not take multiple load levels into account. Existing approaches also fail to leverage publicly available data on server efficiency and instead require experiments to be conducted on the target system. This makes these approaches unwieldy when making decision regarding systems that are not yet available to the decision maker. In this paper, we use the readily available data provided by the SPEC SERT to predict the power consumption of workloads for different load levels in virtualized environments. We evaluate our approach comparing predicted results against measurements of power consumption in multiple virtualized environment configurations on a target server that differs significantly from the reference system used for experimentation. We show that power consumption of CPU and storage loads can be reliably predicted with a prediction error of less than 15% across all tested virtualized environment configurations.}, address = {Cham}, author = {von Kistowski, Joakim and Schreck, Marco and Kounev, Samuel}, booktitle = {Computer Performance Engineering: 13th European Workshop, EPEW 2016, Chios, Greece, October 5-7, 2016, Proceedings}, editor = {Fiems, Dieter and Paolieri, Marco and Platis, N. Agapios}, keywords = {Power}, month = {{October}}, pages = {79–93}, publisher = {Springer International Publishing}, title = {Predicting Power Consumption in Virtualized Environments}, year = 2016 }
• 
  Triggering Performance Counters for Energy Efficiency Measurements. N. Schmitt; (2016, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Schmitt-SSP2016, author = {Schmitt, Norbert}, howpublished = {Experience Talk at SSP 2016}, keywords = {Power}, month = 11, title = {{Triggering Performance Counters for Energy Efficiency Measurements}}, year = 2016 }
• 
  Kieker4DQL: Declarative Performance Measurement. M. Blohm; M. Pahlberg; S. Vogel; J. Walter; D. Okanovic; in Proceedings of the 2016 Symposium on Software Performance (SSP) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The Descartes Query Language (DQL) enables to query the performance of a system using adapters to various solution approaches. Thereby, DQL is a realization of the vision of Declarative Performance Engineering (DPE) which decouples the description of user concerns (performance questions and goals) from the task of selecting and applying a speciffic solution approach. While model-based approaches are already supported by DQL, measurement based approaches are not. In this paper we present the Kieker4DQL, an adapter for DQL that performs query processing for Application Monitoring Tools, the Kieker in particular. It creates a tailored monitoring configuration to start measurements. The resulting monitoring data is filtered and interpreted according to the query.

  @inproceedings{BlPhVoWaOk2016-SPP-Kieker4DQL, abstract = {The Descartes Query Language (DQL) enables to query the performance of a system using adapters to various solution approaches. Thereby, DQL is a realization of the vision of Declarative Performance Engineering (DPE) which decouples the description of user concerns (performance questions and goals) from the task of selecting and applying a speciffic solution approach. While model-based approaches are already supported by DQL, measurement based approaches are not. In this paper we present the Kieker4DQL, an adapter for DQL that performs query processing for Application Monitoring Tools, the Kieker in particular. It creates a tailored monitoring configuration to start measurements. The resulting monitoring data is filtered and interpreted according to the query.}, author = {Blohm, Matthias and Pahlberg, Maksim and Vogel, Sebastian and Walter, J{\"u}rgen and Okanovic, Dusan}, booktitle = {Proceedings of the 2016 Symposium on Software Performance (SSP)}, keywords = {DQL}, month = 11, title = {{ Kieker4DQL: Declarative Performance Measurement}}, year = 2016 }
• 
  Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism. A. Bauer; (2016, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Bauer-SSP2016, author = {Bauer, Andr{\'e}}, howpublished = {Experience Talk at SSP 2016}, keywords = {BUNGEE}, month = 11, title = {{Design and Evaluation of a Proactive, Application-Aware Elasticity Mechanism}}, year = 2016 }
• 
  PAVO: A Framework for the Visualization of Performance Analyses Results. J. Walter; M. König; S. Eismann; S. Kounev; in Proceedings of the 2016 Symposium on Software Performance (SSP) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).

  @inproceedings{WaKoEiKo2016-SPP-PAVO, abstract = {Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).}, author = {Walter, J{\"u}rgen and K{\"o}nig, Maximilian and Eismann, Simon and Kounev, Samuel}, booktitle = {Proceedings of the 2016 Symposium on Software Performance (SSP)}, keywords = {DQL}, month = 11, title = {{ PAVO: A Framework for the Visualization of Performance Analyses Results}}, year = 2016 }
• 
  Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments. A. Milenkoski; K. R. Jayaram; N. Antunes; M. Vieira; S. Kounev; in Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016) (2016).

  Acceptance rate (Full Paper): 45/130 = 34%
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.

  @inproceedings{MiJaAnViKo2016-ISSRE-Quantifying, abstract = {With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.}, address = {{Washington DC, USA}}, author = {Milenkoski, Aleksandar and Jayaram, K. R. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel}, booktitle = {{Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016)}}, keywords = {Virtualization}, month = 10, note = {{Acceptance rate (Full Paper): 45/130 = 34\%}}, organization = {IEEE}, publisher = {IEEE Computer Society}, title = {{Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments}}, year = 2016 }
• 
  Evaluation of Intrusion Detection Systems in Virtualized Environments. A. Milenkoski; Thesis; University of Würzburg, Germany. (2016, November).

  SPEC Kaivalya Dixit Distinguished Dissertation Award 2017
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @phdthesis{Milenkoski2016-phd, author = {Milenkoski, Aleksandar}, keywords = {descartes}, month = 11, note = {{SPEC Kaivalya Dixit Distinguished Dissertation Award 2017}}, school = {University of W{\"u}rzburg, Germany}, title = {Evaluation of Intrusion Detection Systems in Virtualized Environments}, year = 2016 }
• 
  Performance Analysis of SDN Switches with Hardware and Software Flow Tables. P. Rygielski; M. Seliuchenko; S. Kounev; M. Klymash; in Proceedings of the 10th EAI International Conference on Performance Evaluation Methodologies and Tools (ValueTools 2016) (2016).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Nowadays data centers are increasingly becoming larger and dynamic due to virtualization. Software-Defined Networking is the leading approach to network virtualization and flexible management. The wide variety of hardware implementations have brought strong heterogeneity to the market of networking devices which are different in terms of OpenFlow features and performance. In this paper we address the issue of heterogeneity of four hardware OpenFlow switches by characterizing selected performance relevant parameters for the hardware and software flow tables. We characterize maximum size of hardware flow tables for each switch including the behavior of a rule promotion engine that moves the rules between tables. We show that in the worst case forwarding packets using software table decreases the throughput by two orders of magnitude (from 940 to 14 Mbit/s). Our results can help the developers of SDN applications to account performance limitations of hardware and software processing as well as limited hardware support for a specific rule types.

  @inproceedings{RySeKoKl2016-valuetools2016-sdn-switches-hw-sw-flow-tables, abstract = {{Nowadays data centers are increasingly becoming larger and dynamic due to virtualization. Software-Defined Networking is the leading approach to network virtualization and flexible management. The wide variety of hardware implementations have brought strong heterogeneity to the market of networking devices which are different in terms of OpenFlow features and performance. In this paper we address the issue of heterogeneity of four hardware OpenFlow switches by characterizing selected performance relevant parameters for the hardware and software flow tables. We characterize maximum size of hardware flow tables for each switch including the behavior of a rule promotion engine that moves the rules between tables. We show that in the worst case forwarding packets using software table decreases the throughput by two orders of magnitude (from 940 to 14 Mbit/s). Our results can help the developers of SDN applications to account performance limitations of hardware and software processing as well as limited hardware support for a specific rule types. }}, author = {Rygielski, Piotr and Seliuchenko, Marian and Kounev, Samuel and Klymash, Mykhailo}, booktitle = {Proceedings of the 10th EAI International Conference on Performance Evaluation Methodologies and Tools (ValueTools 2016)}, keywords = {DNI}, month = 10, title = {{Performance Analysis of SDN Switches with Hardware and Software Flow Tables}}, year = 2016 }
• 
  Modeling and Prediction of Software-Defined Networks Performance using Queueing Petri Nets. P. Rygielski; M. Seliuchenko; S. Kounev; in Proceedings of the Ninth International Conference on Simulation Tools and Techniques (SIMUTools 2016) (2016). 66–75.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Using various modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. Descartes Network Infrastructure (DNI) is a data center network performance modeling approach that addresses this challenge by offering multiple performance models but requiring to use only a single modeling language. In this paper, we thoroughly extend DNI to support new networking paradigms like, among others, Software-Defined Networking (SDN) and Network-Function Virtualization (NFV). Additionally, we demonstrate how SDN-based networks can be modeled using DNI and how are they transformed later into Queueing Petri Nets (QPN) using a model-to-model transformation. In the analysis of the performance prediction accuracy, we show that automatically generated QPN models represent the performance of heterogeneous SDN hardware with maximal prediction accuracy error of 12 percent.

  @inproceedings{RySeKo2016-SIMUTools-SDN-switches-QPN, abstract = {{Using various modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. Descartes Network Infrastructure (DNI) is a data center network performance modeling approach that addresses this challenge by offering multiple performance models but requiring to use only a single modeling language. In this paper, we thoroughly extend DNI to support new networking paradigms like, among others, Software-Defined Networking (SDN) and Network-Function Virtualization (NFV). Additionally, we demonstrate how SDN-based networks can be modeled using DNI and how are they transformed later into Queueing Petri Nets (QPN) using a model-to-model transformation. In the analysis of the performance prediction accuracy, we show that automatically generated QPN models represent the performance of heterogeneous SDN hardware with maximal prediction accuracy error of 12 percent.}}, author = {Rygielski, Piotr and Seliuchenko, Marian and Kounev, Samuel}, booktitle = {Proceedings of the Ninth International Conference on Simulation Tools and Techniques (SIMUTools 2016)}, keywords = {QPN}, month = {08}, pages = {66–75}, title = {{Modeling and Prediction of Software-Defined Networks Performance using Queueing Petri Nets}}, year = 2016 }
• 
  A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures. S. Kounev; N. Huber; F. Brosig; X. Zhu; in IEEE Computer (2016). 49(7) 53–61.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{KoHuBrZh2016-IEEEComputer-Descartes, author = {Kounev, Samuel and Huber, Nikolaus and Brosig, Fabian and Zhu, Xiaoyun}, journal = {IEEE Computer}, keywords = {Self-adaptive-systems}, month = {07}, number = 7, pages = {53-61}, publisher = {IEEE}, title = {{A Model-Based Approach to Designing Self-Aware IT Systems and Infrastructures}}, volume = 49, year = 2016 }
• 
  Automation and Simplification Through Declarative Performance Engineering. J. Walter; (2016, June).
  • [ BibTeX ]
  • [ Download ]
  @misc{Talk-Kolloquium-2016-06-14, address = {W{\"u}rzburg, Germany}, author = {Walter, J{\"u}rgen}, howpublished = {Doctoral Colloquium W{\"u}rzburg, June 14, 2016}, keywords = {Optimization}, month = {06}, title = {{Automation and Simplification Through Declarative Performance Engineering}}, year = 2016 }
• 
  Benchmarking Dependability of Virtual and Cloud Environments. S. Kounev; (2016, May).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-InvPrague16, author = {Kounev, Samuel}, howpublished = {Invited Talk, Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic}, keywords = {descartes}, month = {05}, title = {{Benchmarking Dependability of Virtual and Cloud Environments}}, year = 2016 }
• 
  The Descartes Modeling Language for Self-Aware Performance and Resource Management. S. Kounev; (Z. M. (Jack) Jiang, ed.) (2016, April).

  IEEE Software Blog
  • [ BibTeX ]
  • [ URL ]
  @misc{DML_IEEESoftwareblog2016, author = {Kounev, Samuel}, editor = {Jiang, Zhen Ming (Jack)}, keywords = {Self-adaptive-systems}, month = {04}, note = {IEEE Software Blog}, title = {{The Descartes Modeling Language for Self-Aware Performance and Resource Management}}, year = 2016 }
• 
  Scientific Computing, Computer Arithmetic, and Validated Numerics: 16th International Symposium, SCAN 2014, Würzburg, Germany, September 21-26, 2014. Revised Selected Papers M. Nehmeier; J. W. von Gudenberg; W. Tucker; (2016). (Vol. 9553) Springer.
  • [ BibTeX ]
  • [ URL ]
  @proceedings{nehmeier2016-SCAN2014-Proceedings, author = {Nehmeier, Marco and von Gudenberg, J{\"u}rgen Wolff and Tucker, Warwick}, keywords = {descartes}, month = {09}, publisher = {Springer}, title = {Scientific Computing, Computer Arithmetic, and Validated Numerics: 16th International Symposium, SCAN 2014, W{\"u}rzburg, Germany, September 21-26, 2014. Revised Selected Papers}, volume = 9553, year = 2016 }

• 
  The Descartes Modeling Language for Self-Aware Performance and Resource Management. S. Kounev; F. Brosig; N. Huber; in Software Engineering 2015, Fachtagung des GI-Fachbereichs Softwaretechnik, 17. - 20. März 2015, Dresden, Deutschland (2015).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KoBrHu2015-SE-SelfAwareDescartes, address = {Bonn, Germany}, author = {Kounev, Samuel and Brosig, Fabian and Huber, Nikolaus}, booktitle = {Software Engineering 2015, Fachtagung des GI-Fachbereichs Softwaretechnik, 17. - 20. M{\"a}rz 2015, Dresden, Deutschland}, keywords = {Self-aware-computing}, month = {03}, publisher = {GI}, series = {Lecture Notes in Informatics (LNI)}, title = {{The Descartes Modeling Language for Self-Aware Performance and Resource Management}}, year = 2015 }
• 
  Load Testing Elasticity and Performance Isolation in Shared Execution Environments. S. Kounev; (2015, February).
  • [ BibTeX ]
  @misc{talk-Kounev-LT2015, author = {Kounev, Samuel}, howpublished = {Keynote Talk at the 4th International Workshop on Large-Scale Testing (LT 2015), co-located with ICPE 2015, Austin, TX, USA, February 1, 2015}, keywords = {Talks}, month = {02}, title = {{Load Testing Elasticity and Performance Isolation in Shared Execution Environments.}}, year = 2015 }
• 
  Load Testing Elasticity and Performance Isolation in Shared Execution Environments. S. Kounev; (2015, February).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-LT15, author = {Kounev, Samuel}, howpublished = {Keynote Talk at the 4th International Workshop on Large-Scale Testing (LT 2015), co-located with ICPE 2015, Austin, TX, USA}, keywords = {Talks}, month = {02}, title = {{Load Testing Elasticity and Performance Isolation in Shared Execution Environments}}, year = 2015 }
• 
  The Descartes Modeling Language for Self-Aware Performance and Resource Management. S. Kounev; (2015, March).
  • [ BibTeX ]
  • [ Download ]
  @misc{Talk-Ko2015-SE-SelfAwareDescartes, author = {Kounev, Samuel}, howpublished = {Software Engineering 2015, Fachtagung des GI-Fachbereichs Softwaretechnik, 17. - 20. M{\"a}rz 2015, Dresden, Germany}, keywords = {Talks}, month = {03}, title = {{The Descartes Modeling Language for Self-Aware Performance and Resource Management}}, year = 2015 }
• 
  Self-Aware Performance and Resource Management in Shared IT Infrastructures. S. Kounev; (2015, May).
  • [ BibTeX ]
  @misc{talk-Kounev-DATEV2015, author = {Kounev, Samuel}, howpublished = {Invited Talk, DATEV eG, 90329 N{\"u}rnberg, Germany}, keywords = {Talks}, month = {05}, title = {{Self-Aware Performance and Resource Management in Shared IT Infrastructures}}, year = 2015 }
• 
  BUNGEE: An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments. N. R. Herbst; S. Kounev; A. Weber; H. Groenda; in Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015) (2015).

  Acceptance rate: 29%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Today's infrastructure clouds provide resource elasticity (i.e. auto-scaling) mechanisms enabling self-adaptive resource provisioning to reflect variations in the load intensity over time. These mechanisms impact on the application performance, however, their effect in specific situations is hard to quantify and compare. To evaluate the quality of elasticity mechanisms provided by different platforms and configurations, respective metrics and benchmarks are required. Existing metrics for elasticity only consider the time required to provision and deprovision resources or the costs impact of adaptations. Existing benchmarks lack the capability to handle open workloads with realistic load intensity profiles and do not explicitly distinguish between the performance exhibited by the provisioned underlying resources, on the one hand, and the quality of the elasticity mechanisms themselves, on the other hand. In this paper, we propose reliable metrics for quantifying the timing aspects and accuracy of elasticity. Based on these metrics, we propose a novel approach for benchmarking the elasticity of Infrastructure-as-a-Service (IaaS) cloud platforms independent of the performance exhibited by the provisioned underlying resources. We show that the proposed metrics provide consistent ranking of elastic platforms on an ordinal scale. Finally, we present an extensive case study of real-world complexity demonstrating that the proposed approach is applicable in realistic scenarios and can cope with different levels of resource efficiency.

  @inproceedings{HeKoWeGr_2015_SEAMS_BUNGEE, abstract = {Today's infrastructure clouds provide resource elasticity (i.e. auto-scaling) mechanisms enabling self-adaptive resource provisioning to reflect variations in the load intensity over time. These mechanisms impact on the application performance, however, their effect in specific situations is hard to quantify and compare. To evaluate the quality of elasticity mechanisms provided by different platforms and configurations, respective metrics and benchmarks are required. Existing metrics for elasticity only consider the time required to provision and deprovision resources or the costs impact of adaptations. Existing benchmarks lack the capability to handle open workloads with realistic load intensity profiles and do not explicitly distinguish between the performance exhibited by the provisioned underlying resources, on the one hand, and the quality of the elasticity mechanisms themselves, on the other hand. In this paper, we propose reliable metrics for quantifying the timing aspects and accuracy of elasticity. Based on these metrics, we propose a novel approach for benchmarking the elasticity of Infrastructure-as-a-Service (IaaS) cloud platforms independent of the performance exhibited by the provisioned underlying resources. We show that the proposed metrics provide consistent ranking of elastic platforms on an ordinal scale. Finally, we present an extensive case study of real-world complexity demonstrating that the proposed approach is applicable in realistic scenarios and can cope with different levels of resource efficiency.}, author = {Herbst, Nikolas Roman and Kounev, Samuel and Weber, Andreas and Groenda, Henning}, booktitle = {Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015)}, keywords = {LIMBO}, month = {05}, note = {Acceptance rate: 29\%}, title = {{BUNGEE: An Elasticity Benchmark for Self-Adaptive IaaS Cloud Environments}}, year = 2015 }
• 
  Modeling and Extracting Load Intensity Profiles. J. von Kistowski; N. R. Herbst; D. Zoller; S. Kounev; A. Hotho; in Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015) (2015).

  Acceptance rate: 29%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Benchmarking of systems under these constraints is difficult, as state-of-the-art benchmarking frameworks provide only limited support for emulating such dynamic and highly variable load profiles for the creation of realistic workload scenarios. Industrial benchmarks typically confine themselves to workloads with constant or stepwise increasing loads. Alternatively, they support replaying of recorded load traces. Statistical load intensity descriptions also do not sufficiently capture concrete pattern load profile variations over time. To address these issues, we present the Descartes Load Intensity Model (DLIM). DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance can be used as a compact representation of a recorded load intensity trace, providing a powerful tool for benchmarking and performance analysis. As manually obtaining DLIM instances can be time consuming, we present three different automated extraction methods, which also help to enable autonomous system analysis for self-adaptive systems. Model expressiveness is validated using the presented extraction methods. Extracted DLIM instances exhibit a median modeling error of 12.4% on average over nine different real-world traces covering between two weeks and seven months. Additionally, extraction methods perform orders of magnitude faster than existing time series decomposition approaches.

  @inproceedings{KiHeZoKoHo2015_SEAMS_DLIMExtraction, abstract = {Today's system developers and operators face the challenge of creating software systems that make efficient use of dynamically allocated resources under highly variable and dynamic load profiles, while at the same time delivering reliable performance. Benchmarking of systems under these constraints is difficult, as state-of-the-art benchmarking frameworks provide only limited support for emulating such dynamic and highly variable load profiles for the creation of realistic workload scenarios. Industrial benchmarks typically confine themselves to workloads with constant or stepwise increasing loads. Alternatively, they support replaying of recorded load traces. Statistical load intensity descriptions also do not sufficiently capture concrete pattern load profile variations over time. To address these issues, we present the Descartes Load Intensity Model (DLIM). DLIM provides a modeling formalism for describing load intensity variations over time. A DLIM instance can be used as a compact representation of a recorded load intensity trace, providing a powerful tool for benchmarking and performance analysis. As manually obtaining DLIM instances can be time consuming, we present three different automated extraction methods, which also help to enable autonomous system analysis for self-adaptive systems. Model expressiveness is validated using the presented extraction methods. Extracted DLIM instances exhibit a median modeling error of 12.4% on average over nine different real-world traces covering between two weeks and seven months. Additionally, extraction methods perform orders of magnitude faster than existing time series decomposition approaches.}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas Roman and Zoller, Daniel and Kounev, Samuel and Hotho, Andreas}, booktitle = {Proceedings of the 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2015)}, keywords = {LIMBO}, month = {05}, note = {Acceptance rate: 29\%}, title = {{Modeling and Extracting Load Intensity Profiles}}, year = 2015 }
• 
  How elastic is your cloud? From metrics to benchmarks. S. Kounev; (2015, May).
  • [ BibTeX ]
  @misc{talk-Kounev-DATEV_TrendScout2015a, author = {Kounev, Samuel}, howpublished = {Invited Talk, DATEV Trendscout, N{\"u}rnberg, Germany}, keywords = {Talks}, month = {05}, title = {{How elastic is your cloud? From metrics to benchmarks}}, year = 2015 }
• 
  Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments. A. Busch; Q. Noorshams; S. Kounev; A. Koziolek; R. Reussner; E. Amrehn; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2015).

  Acceptance Rate (Full Paper): 15/56 = 27%.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{BuNoKoKoReAm2015-ICPE-WorkloadCharacterization, address = {New York, NY, USA}, author = {Busch, Axel and Noorshams, Qais and Kounev, Samuel and Koziolek, Anne and Reussner, Ralf and Amrehn, Erich}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {se2}, month = {02}, note = {Acceptance Rate (Full Paper): 15/56 = 27\%.}, publisher = {ACM}, series = {ICPE '15}, title = {{Automated Workload Characterization for I/O Performance Analysis in Virtualized Environments}}, year = 2015 }
• 
  Engineering & Provisioning of Cloud Applications - Research and Entrepreneurship. S. Kounev; (2015, February).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-WsWue15, author = {Kounev, Samuel}, howpublished = {Winder School, University of W{\"u}rzburg, Germany}, keywords = {Talks}, month = {02}, title = {{Engineering & Provisioning of Cloud Applications - Research and Entrepreneurship}}, year = 2015 }
• 
  Model-driven Algorithms and Architectures for Self-Aware Computing Systems S. Kounev; in Dagstuhl Reports, (S. Kounev; X. Zhu; J. O. Kephart; M. Kwiatkowska; A. M. [Assistant Editor], eds.) (2015). Dagstuhl, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.

  @proceedings{KoZhKeKw2015, abstract = {{This report documents the program and the outcomes of Dagstuhl Seminar 15041 "Model-driven Algorithms and Architectures for Self-Aware Computing Systems". The design of self-aware computing systems calls for an integrated interdisciplinary approach building on results from multiple areas of computer science and engineering, including software and systems engineering, systems modeling, simulation and analysis, autonomic and organic computing, machine learning and artificial intelligence, data center resource management, and so on. The Dagstuhl Seminar 15041 served as a platform to raise the awareness about the relevant research efforts in the respective research communities as well as existing synergies that can be exploited to advance the state-of-the-art, formulate a new research agenda that takes a broader view on the problem following an integrated and interdisciplinary approach, and establish collaborations between academia and industry.}}, address = {Dagstuhl, Germany}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Zhu, Xiaoyun and Kephart, Jeffrey O. and Kwiatkowska, Marta and Editor], Aleksandar Milenkoski [Assistant}, keywords = {Self-adaptive-systems}, month = {01}, series = {Dagstuhl Reports}, title = {{Model-driven Algorithms and Architectures for Self-Aware Computing Systems}}, year = 2015 }
• 
  The Storage Performance Analyzer: Measuring, Monitoring, and Modeling of I/O Performance in Virtualized Environments. Q. Noorshams; A. Busch; S. Kounev; R. Reussner; in Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (2015).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoBuKoRe2015-ICPE-InvitedDemo, author = {Noorshams, Qais and Busch, Axel and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering}, keywords = {Virtualization}, month = {02}, series = {ICPE '15}, title = {{The Storage Performance Analyzer: Measuring, Monitoring, and Modeling of I/O Performance in Virtualized Environments}}, year = 2015 }
• 
  The SPEC Research Group - Status Quo. S. Kounev; (2015, February).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-SPEC15, author = {Kounev, Samuel}, howpublished = {Talk at the SPEC 2015 Annual Meeting, Austin, Texas, USA}, keywords = {SPEC}, month = {02}, title = {{The SPEC Research Group - Status Quo}}, year = 2015 }
• 
  Load Testing Elasticity and Performance Isolation in Shared Execution Environments. S. Kounev; in Proceedings of the 4th International Workshop on Large-Scale Testing (LT 2015), co-located with the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015) (2015).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{Ko2015-ICPE_LT_Workshop-Keynote, address = {New York, NY, USA}, author = {Kounev, Samuel}, booktitle = {Proceedings of the 4th International Workshop on Large-Scale Testing (LT 2015), co-located with the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015)}, keywords = {Isolation}, month = {02}, publisher = {ACM}, title = {{Load Testing Elasticity and Performance Isolation in Shared Execution Environments}}, year = 2015 }
• 
  How to Build a Benchmark. J. von Kistowski; J. A. Arnold; K. Huppler; K.-D. Lange; J. L. Henning; P. Cao; in Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Standardized benchmarks have become widely accepted tools for the comparison of products and evaluation of methodologies. These benchmarks are created by consortia like SPEC and TPC under confidentiality agreements which provide little opportunity for outside observers to get a look at the processes and concerns that are prevalent in benchmark development. This paper introduces the primary concerns of benchmark development from the perspectives of SPEC and TPC committees. We provide a benchmark definition, outline the types of benchmarks, and explain the characteristics of a good benchmark. We focus on the characteristics important for a standardized benchmark, as created by the SPEC and TPC consortia. To this end, we specify the primary criteria to be employed for benchmark design and workload selection. We use multiple standardized benchmarks as examples to demonstrate how these criteria are ensured.

  @inproceedings{KiArHuLaHeCa2015-ICPE-Benchmark, abstract = {{Standardized benchmarks have become widely accepted tools for the comparison of products and evaluation of methodologies. These benchmarks are created by consortia like SPEC and TPC under confidentiality agreements which provide little opportunity for outside observers to get a look at the processes and concerns that are prevalent in benchmark development. This paper introduces the primary concerns of benchmark development from the perspectives of SPEC and TPC committees. We provide a benchmark definition, outline the types of benchmarks, and explain the characteristics of a good benchmark. We focus on the characteristics important for a standardized benchmark, as created by the SPEC and TPC consortia. To this end, we specify the primary criteria to be employed for benchmark design and workload selection. We use multiple standardized benchmarks as examples to demonstrate how these criteria are ensured.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Arnold, Jeremy A. and Huppler, Karl and Lange, Klaus-Dieter and Henning, John L. and Cao, Paul}, booktitle = {Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015)}, keywords = {Power}, month = {02}, publisher = {ACM}, series = {ICPE '15}, title = {{How to Build a Benchmark}}, year = 2015 }
• 
  Modeling and Prediction of I/O Performance in Virtualized Environments. Q. Noorshams; Thesis; Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany. (2015, February).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @phdthesis{Noorshams2015-phd, author = {Noorshams, Qais}, keywords = {descartes}, month = {02}, school = {Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany}, title = {Modeling and Prediction of I/O Performance in Virtualized Environments}, year = 2015 }
• 
  Analysis of the Influences on Server Power Consumption and Energy Efficiency for CPU-Intensive Workloads. J. von Kistowski; H. Block; J. Beckett; K.-D. Lange; J. A. Arnold; S. Kounev; in Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015) (2015).

  acceptance rate: 27%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Energy efficiency of servers has become a significant research topic over the last years, as server energy consumption varies depending on multiple factors, such as server utilization and workload type. Server energy analysis and estimation must take all relevant factors into account to ensure reliable estimates and conclusions. Thorough system analysis requires benchmarks capable of testing different system resources at different load levels using multiple workload types. Server energy estimation approaches, on the other hand, require knowledge about the interactions of these factors for the creation of accurate power models. Common approaches to energy-aware workload classification classify workloads depending on the resource types used by the different workloads. However, they rarely take into account differences in workloads targeting the same resources. Industrial energy-efficiency benchmarks typically do not evaluate the system's energy consumption at different resource load levels, and they only provide data for system analysis at maximum system load. In this paper, we benchmark multiple server configurations using the CPU worklets included in SPEC's Server Efficiency Rating Tool (SERT). We evaluate the impact of load levels and different CPU workloads on power consumption and energy efficiency. We analyze how functions approximating the measured power consumption differ over multiple server configurations and architectures. We show that workloads targeting the same resource can differ significantly in their power draw and energy efficiency. The power consumption of a given workload type varies depending on utilization, hardware and software configuration. The power consumption of CPU-intensive workloads does not scale uniformly with increased load, nor do hardware or software configuration changes affect it in a uniform manner.

  @inproceedings{KiBlBeLaArKo2015-ICPE-SERT, abstract = {{ Energy efficiency of servers has become a significant research topic over the last years, as server energy consumption varies depending on multiple factors, such as server utilization and workload type. Server energy analysis and estimation must take all relevant factors into account to ensure reliable estimates and conclusions. Thorough system analysis requires benchmarks capable of testing different system resources at different load levels using multiple workload types. Server energy estimation approaches, on the other hand, require knowledge about the interactions of these factors for the creation of accurate power models. Common approaches to energy-aware workload classification classify workloads depending on the resource types used by the different workloads. However, they rarely take into account differences in workloads targeting the same resources. Industrial energy-efficiency benchmarks typically do not evaluate the system's energy consumption at different resource load levels, and they only provide data for system analysis at maximum system load. In this paper, we benchmark multiple server configurations using the CPU worklets included in SPEC's Server Efficiency Rating Tool (SERT). We evaluate the impact of load levels and different CPU workloads on power consumption and energy efficiency. We analyze how functions approximating the measured power consumption differ over multiple server configurations and architectures. We show that workloads targeting the same resource can differ significantly in their power draw and energy efficiency. The power consumption of a given workload type varies depending on utilization, hardware and software configuration. The power consumption of CPU-intensive workloads does not scale uniformly with increased load, nor do hardware or software configuration changes affect it in a uniform manner.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Block, Hansfried and Beckett, John and Lange, Klaus-Dieter and Arnold, Jeremy A. and Kounev, Samuel}, booktitle = {Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering (ICPE 2015)}, keywords = {Power}, month = {02}, note = {acceptance rate: 27\%}, publisher = {ACM}, series = {ICPE '15}, title = {{Analysis of the Influences on Server Power Consumption and Energy Efficiency for CPU-Intensive Workloads}}, year = 2015 }
• 
  Software Engineering for Self-Aware Computing. S. Kounev; (2015, January).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Dagstuhl15, author = {Kounev, Samuel}, howpublished = {Tutorial Talk at Dagstuhl Seminar 15041 on ''Model-driven Algorithms and Architectures for Self-Aware Computing Systems''}, keywords = {Talks}, month = {01}, title = {{Software Engineering for Self-Aware Computing}}, year = 2015 }
• 
  The Descartes Modeling Language and Tool Chain. S. Kounev; (2015, January).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Dagstuhl15-2, author = {Kounev, Samuel}, howpublished = {Presentation at Dagstuhl Seminar 15041 on ''Model-driven Algorithms and Architectures for Self-Aware Computing Systems''}, keywords = {descartes}, month = {01}, title = {{The Descartes Modeling Language and Tool Chain}}, year = 2015 }
• 
  Software Engineering Research at the University of Würzburg. S. Kounev; (2015, June).
  • [ BibTeX ]
  @misc{talk-Kounev-CAS2015, author = {Kounev, Samuel}, howpublished = {Invited Talk, CAS Software AG, Karlsruhe, Germany}, keywords = {Talks}, month = {06}, title = {{Software Engineering Research at the University of W\"urzburg}}, year = 2015 }
• 
  SPEC Research Group Newsletter, vol. 1 no. 4. S. Kounev; K. Sachs; P. Rygielski; (2015, January).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-4, author = {Kounev, Samuel and Sachs, Kai and Rygielski, Piotr}, keywords = {SPEC}, month = {01}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 1 no. 4}}, year = 2015 }
• 
  PABS-a Programming Assignment Feedback System. L. Iffländer; A. Dallmann; P.-D. Beck; M. Ifland; in ABP (2015).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{ifflander2015pabs, author = {Iffländer, Lukas and Dallmann, Alexander and Beck, Philip-Daniel and Ifland, Marianus}, booktitle = {ABP}, keywords = {t_workshop}, title = {PABS-a Programming Assignment Feedback System.}, year = 2015 }
• 
  Proactive Memory Scaling of Virtualized Applications. S. Spinner; N. Herbst; S. Kounev; X. Zhu; L. Lu; M. Uysal; R. Griffith; in Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015) (2015). 277–284.

  Acceptance Rate: 15%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.

  @inproceedings{SpHeKoZhLuUyGr2015-CLOUD-ProactiveMemoryScaling, abstract = {{Enterprise applications in virtualized environments are often subject to time-varying workloads with multiple seasonal patterns and trends. In order to ensure quality of service for such applications while avoiding over-provisioning, resources need to be dynamically adapted to accommodate the current workload demands. Many memory-intensive applications are not suitable for the traditional horizontal scaling approach often used for runtime performance management, as it relies on complex and expensive state replication. On the other hand, vertical scaling of memory often requires a restart of the application. In this paper, we propose a proactive approach to memory scaling for virtualized applications. It uses statistical forecasting to predict the future workload and reconfigure the memory size of the virtual machine of an application automatically. To this end, we propose an extended forecasting technique that leverages meta-knowledge, such as calendar information, to improve the forecast accuracy. In addition, we develop an application controller to adjust settings associated with application memory management during memory reconfiguration. Our evaluation using real-world traces shows that the forecast accuracy quantified with the MASE error metric can be improved by 11-59%. Furthermore, we demonstrate that the proactive approach can reduce the impact of reconfiguration on application availability by over 80% and significantly improve performance relative to a reactive controller.}}, author = {Spinner, Simon and Herbst, Nikolas and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Griffith, Rean}, booktitle = {Proceedings of the 2015 IEEE 8th International Conference on Cloud Computing (IEEE CLOUD 2015)}, keywords = {Self-adaptive-systems}, month = {06}, note = {Acceptance Rate: 15%}, pages = {277–284}, publisher = {IEEE}, title = {{Proactive Memory Scaling of Virtualized Applications}}, year = 2015 }
• 
  Automated Transformation of Descartes Modeling Language to Palladio Component Model. J. Walter; S. Eismann; A. Hildebrandt; in Proceedings of the 2015 Symposium on Software Performance (SSP) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Model-based performance predictions and reconfigurations enable optimizing resource efficiency while ensuring that Quality-of-Service demands are met in today's complex ITsystems. The Descartes Modeling Language (DML) and the Palladio Component Model (PCM) are two architectural performance modeling formalisms applied in this context. This paper compares DML to PCM concerning similarities, differences and semantic gaps. Based on this, we propose a mapping from DML to PCM for which we implemented a tool realizing an automated transformation.

  @inproceedings{WaEiHi2015-SPP-DML2PCM, abstract = {Model-based performance predictions and reconfigurations enable optimizing resource efficiency while ensuring that Quality-of-Service demands are met in today's complex ITsystems. The Descartes Modeling Language (DML) and the Palladio Component Model (PCM) are two architectural performance modeling formalisms applied in this context. This paper compares DML to PCM concerning similarities, differences and semantic gaps. Based on this, we propose a mapping from DML to PCM for which we implemented a tool realizing an automated transformation.}, author = {Walter, J{\"u}rgen and Eismann, Simon and Hildebrandt, Adrian}, booktitle = {Proceedings of the 2015 Symposium on Software Performance (SSP)}, keywords = {t_workshop}, month = 11, title = {{Automated Transformation of Descartes Modeling Language to Palladio Component Model}}, year = 2015 }
• 
  Challenges of Assessing the Hypercall Interface Robustness. D. Carvalho; N. Antunes; M. Vieira; A. Milenkoski; S. Kounev; in The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.

  @inproceedings{CaAnViMiKo2015-DSN-Challenges, abstract = {Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.}, author = {Carvalho, Diogo and Antunes, Nuno and Vieira, Marco and Milenkoski, Aleksandar and Kounev, Samuel}, booktitle = {The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015)}, keywords = {Reliability}, month = {06}, publisher = {{IEEE}}, title = {{Challenges of Assessing the Hypercall Interface Robustness}}, year = 2015 }
• 
  Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study. N. Huber; J. Walter; M. Bähr; S. Kounev; in Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.

  @inproceedings{HuWaBaKo2015-ICCAC-BlueYonder, abstract = {Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.}, author = {Huber, Nikolaus and Walter, J{\"u}rgen and B{\"a}hr, Manuel and Kounev, Samuel}, booktitle = {Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC)}, keywords = {Self-adaptive-systems}, month = {09}, publisher = {IEEE}, title = {{Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study}}, year = 2015 }
• 
  Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices. A. Milenkoski; M. Vieira; S. Kounev; A. Avritzer; B. D. Payne; in ACM Computing Surveys (2015). 48(1) 12:1–12:41.

  5-year Impact Factor (2014): 5.949
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.

  @article{MiViKoAvPa2015-CSUR-IDSEval, abstract = {{The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.}}, address = {New York, NY, USA}, author = {Milenkoski, Aleksandar and Vieira, Marco and Kounev, Samuel and Avritzer, Alberto and Payne, Bryan D.}, journal = {{ACM Computing Surveys}}, keywords = {Reliability}, month = {{September}}, note = {{<b>5-year Impact Factor (2014): 5.949</b>}}, number = 1, pages = {12:1–12:41}, publisher = {ACM}, title = {{Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices}}, volume = 48, year = 2015 }
• 
  Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection. A. Milenkoski; B. D. Payne; N. Antunes; M. Vieira; S. Kounev; A. Avritzer; M. Luft; in The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015) (2015).

  Acceptance Rate: 23%.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.

  @inproceedings{MiPaAnViKoAvLu2015-RAID-Challenges, abstract = {{The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.}}, author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel and Avritzer, Alberto and Luft, Matthias}, booktitle = {The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015)}, keywords = {Virtualization}, month = {{November}}, note = {{Acceptance Rate: 23%.}}, publisher = {{Springer}}, title = {{Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection}}, year = 2015 }
• 
  ITF1788: An Interval Testframework for IEEE 1788. (495), M. Kiesner; M. Nehmeier; J. Wolff von Gudenberg; (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Libraries implementing the interval arithmetic standard IEEE 1788 have to be tested for errors. Although the standard is language agnostic, the different implementations are not. Therefore the same tests must be implemented in many different languages. Hence it is useful to only have to formulate the tests once in a domain-specific language and to subsequently generate the language-specific test code automatically. To save time and encourage test-driven development, we present a language for testing interval arithmetic libraries and a corresponding source code generator which produces language-specific unit tests.

  @techreport{nehmeierTR495, abstract = {Libraries implementing the interval arithmetic standard IEEE 1788 have to be tested for errors. Although the standard is language agnostic, the different implementations are not. Therefore the same tests must be implemented in many different languages. Hence it is useful to only have to formulate the tests once in a domain-specific language and to subsequently generate the language-specific test code automatically. To save time and encourage test-driven development, we present a language for testing interval arithmetic libraries and a corresponding source code generator which produces language-specific unit tests.}, author = {Kiesner, Maximilian and Nehmeier, Marco and Wolff von Gudenberg, J{\"u}rgen}, institution = {Department of Computer Science, University of W{\"u}rzburg}, keywords = {se2}, month = {06}, number = 495, title = {{ITF1788: An Interval Testframework for IEEE 1788}}, year = 2015 }
• 
  Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques. F. Willnecker; M. Dlugi; A. Brunnert; S. Spinner; S. Kounev; H. Krcmar; in Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015), M. Beltrán, W. Knottenbelt, J. Bradley (eds.) (2015). (Vol. 9272) 115–129.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.

  @inproceedings{WiDlBrSpKoKr2015-EPEW-LibredePMWT, abstract = {Resource demands are a core aspect of performance models. They describe how an operation utilizes a resource and therefore influence the systems performance metrics: response time, resource utilization and throughput. Such demands can be determined by two extraction classes: direct measurement or demand estimation. Selecting the best suited technique depends on available tools, acceptable measurement overhead and the level of granularity necessary for the performance model. This work compares two direct measurement techniques and an adaptive estimation technique based on multiple statistical approaches to evaluate strengths and weaknesses of each technique.We conduct a series of experiments using the SPECjEnterprise2010 industry benchmark and an automatic performance model generator for architecture-level performance models based on the Palladio Component Model. To compare the techniques we conduct two experiments with different levels of granularity on a standalone system, followed by one experiment using a distributed SPECjEnterprise2010 deployment combining both extraction classes for generating a full-stack performance model.}, author = {Willnecker, Felix and Dlugi, Markus and Brunnert, Andreas and Spinner, Simon and Kounev, Samuel and Krcmar, Helmut}, booktitle = {Computer Performance Engineering - Proceedings of the 12th European Workshop (EPEW 2015)}, editor = {Beltr{\'a}n, Marta and Knottenbelt, William and Bradley, Jeremy}, keywords = {t_workshop}, month = {{August}}, pages = {115-129}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{Comparing the Accuracy of Resource Demand Measurement and Estimation Techniques}}, volume = 9272, year = 2015 }
• 
  Univariate Interpolation-based Modeling of Power and Performance. J. von Kistowski; S. Kounev; in Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2015) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Performance and power scale non-linearly with device utilization, making characterization and prediction of energy efficiency at a given load level a challenging issue. A common approach to address this problem is the creation of power or performance state tables for a pre-measured subset of all possible system states. Approaches to determine performance and power for a state not included in the measured subset use simple interpolation, such as nearest neighbor interpolation, or define state switching rules. This leads to a loss in accuracy, as unmeasured system states are not considered. In this paper, we compare different interpolation functions and automatically configure and select functions for a given domain or measurement set. We evaluate our approach by comparing interpolation of measurement data subsets against power and performance measurements on a commodity server. We show that for non-extrapolating models interpolation is significantly more accurate than regression, with our automatically configured interpolation function improving modeling accuracy up to 43.6%.}

  @inproceedings{KiKo2015-VALUETOOLS, abstract = {{Performance and power scale non-linearly with device utilization, making characterization and prediction of energy efficiency at a given load level a challenging issue. A common approach to address this problem is the creation of power or performance state tables for a pre-measured subset of all possible system states. Approaches to determine performance and power for a state not included in the measured subset use simple interpolation, such as nearest neighbor interpolation, or define state switching rules. This leads to a loss in accuracy, as unmeasured system states are not considered. In this paper, we compare different interpolation functions and automatically configure and select functions for a given domain or measurement set. We evaluate our approach by comparing interpolation of measurement data subsets against power and performance measurements on a commodity server. We show that for non-extrapolating models interpolation is significantly more accurate than regression, with our automatically configured interpolation function improving modeling accuracy up to 43.6\%.}}, author = {von Kistowski, J{\'o}akim and Kounev, Samuel}, booktitle = {Proceedings of the 9th EAI International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2015)}, keywords = {Power}, month = 12, title = {{Univariate Interpolation-based Modeling of Power and Performance}}, year = 2015 }
• 
  Industrie 4.1: So wird die Produktion wirklich smart!. S. Kounev; (2015, November).
  • [ BibTeX ]
  @misc{talk-Kounev-DATEV_TrendScout2015b, author = {Kounev, Samuel}, howpublished = {Invited Talk, DATEV Trendscout, N{\"u}rnberg, Germany}, keywords = {Talks}, month = 11, title = {{Industrie 4.1: So wird die Produktion wirklich smart!}}, year = 2015 }
• 
  Resource Demand Estimation in Distributed, Service-Oriented Applications using LibReDE. S. Spinner; (2015, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Spinner-SSP15, author = {Spinner, Simon}, howpublished = {Technical Talk at Symposion on Software Performance (SSP) 2015}, keywords = {descartes}, month = 11, title = {{Resource Demand Estimation in Distributed, Service-Oriented Applications using LibReDE}}, year = 2015 }
• 
  Common Errors and Assumptions in Energy Measurement and Management. J. von Kistowski; (2015, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kistowski-SSP15, author = {von Kistowski, J{\'o}akim}, howpublished = {Technical Talk at Symposium on Software Performance (SSP) 2015}, keywords = {Power}, month = 11, title = {{Common Errors and Assumptions in Energy Measurement and Management}}, year = 2015 }
• 
  Energy Efficiency of Hierarchical Server Load Distribution Strategies. J. von Kistowski; J. Beckett; K.-D. Lange; H. Block; J. A. Arnold; S. Kounev; in Proceedings of the IEEE 23nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2015) (2015).

  Full paper acceptance rate: 19%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Energy efficiency of servers has become a significant issue over the last years. Load distribution plays a crucial role in the improvement of energy efficiency as (un-)balancing strategies can be leveraged to distribute load over one or multiple systems in a way in which resources are utilized at high performance, yet low overall power consumption. This can be achieved on multiple levels, from load distribution on single CPU cores to machine level load balancing on distributed systems. With modern day server architectures providing load balancing opportunities at several layers, answering the question of optimal load distribution has become non-trivial. Work has to be distributed hierarchically in a fashion that enables maximum energy efficiency at each level. Current approaches balance load based on generalized assumptions about the energy efficiency of servers. These assumptions are based either on very machine-specific or highly generalized observations that may or may not hold true over a variety of systems and configurations. In this paper, we use a modified version of the SPEC SERT suite to measure the energy efficiency of a variety of hierarchical load distribution strategies on single and multi-node systems. We introduce a new strategy and evaluate energy efficiency for homogeneous and heterogeneous workloads over different hardware configurations. Our results show that the selection of a load distribution strategy depends heavily on workload, system utilization, as well as hardware. Used in conjunction with existing strategies, our new load distribution strategy can reduce a single system's power consumption by up to 10.7%.

  @inproceedings{KiBeLaBlArKo2015-MASCOTS, abstract = {{Energy efficiency of servers has become a significant issue over the last years. Load distribution plays a crucial role in the improvement of energy efficiency as (un-)balancing strategies can be leveraged to distribute load over one or multiple systems in a way in which resources are utilized at high performance, yet low overall power consumption. This can be achieved on multiple levels, from load distribution on single CPU cores to machine level load balancing on distributed systems. With modern day server architectures providing load balancing opportunities at several layers, answering the question of optimal load distribution has become non-trivial. Work has to be distributed hierarchically in a fashion that enables maximum energy efficiency at each level. Current approaches balance load based on generalized assumptions about the energy efficiency of servers. These assumptions are based either on very machine-specific or highly generalized observations that may or may not hold true over a variety of systems and configurations. In this paper, we use a modified version of the SPEC SERT suite to measure the energy efficiency of a variety of hierarchical load distribution strategies on single and multi-node systems. We introduce a new strategy and evaluate energy efficiency for homogeneous and heterogeneous workloads over different hardware configurations. Our results show that the selection of a load distribution strategy depends heavily on workload, system utilization, as well as hardware. Used in conjunction with existing strategies, our new load distribution strategy can reduce a single system's power consumption by up to 10.7%.}}, author = {von Kistowski, J{\'o}akim and Beckett, John and Lange, Klaus-Dieter and Block, Hansfried and Arnold, Jeremy A. and Kounev, Samuel}, booktitle = {Proceedings of the IEEE 23nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2015)}, keywords = {Power}, month = 10, note = {Full paper acceptance rate: 19\%}, publisher = {IEEE}, title = {{Energy Efficiency of Hierarchical Server Load Distribution Strategies}}, year = 2015 }
• 
  Evaluating Approaches to Resource Demand Estimation. S. Spinner; G. Casale; F. Brosig; S. Kounev; in Performance Evaluation (2015). 92 51–71.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.

  @article{SpCaBrKo2015-PEVA-ResDemEstSurvey, abstract = {Resource demands are a key parameter of stochastic performance models that needs to be determined when performing a quantitative performance analysis of a system. However, the direct measurement of resource demands is not feasible in most realistic systems. Therefore, statistical approaches that estimate resource demands based on coarse-grained monitoring data (e.g., CPU utilization, and response times) have been proposed in the literature. These approaches have different assumptions and characteristics that need to be considered when estimating resource demands. This paper surveys the state-of-the-art in resource demand estimation and proposes a classification scheme for estimation approaches. Furthermore, it contains an experimental evaluation comparing the impact of different factors (monitoring window size, number of workload classes, load level, collinearity, and model mismatch) on the estimation accuracy of seven different approaches. The classification scheme and the experimental comparison helps performance engineers to select an approach to resource demand estimation that fulfills the requirements of a given analysis scenario.}, author = {Spinner, Simon and Casale, Giuliano and Brosig, Fabian and Kounev, Samuel}, journal = {Performance Evaluation}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = 10, pages = {51–71}, publisher = {Elsevier B.V.}, title = {{Evaluating Approaches to Resource Demand Estimation}}, volume = 92, year = 2015 }
• 
  Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures. F. Brosig; P. Meier; S. Becker; A. Koziolek; H. Koziolek; S. Kounev; in IEEE Transactions on Software Engineering (TSE) (2015). 41(2) 157–175.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{BrMeBeKoKoKo2014-TSE-ModelSolutionTradeOffs, author = {Brosig, Fabian and Meier, Philipp and Becker, Steffen and Koziolek, Anne and Koziolek, Heiko and Kounev, Samuel}, journal = {IEEE Transactions on Software Engineering (TSE)}, keywords = {QPN}, month = {{February}}, number = 2, pages = {157–175}, publisher = {IEEE}, title = {{Quantitative Evaluation of Model-Driven Performance Analysis and Simulation of Component-based Architectures}}, volume = 41, year = 2015 }
• 
  Specialized Heuristics for the Controller Placement Problem in Large Scale SDN Networks. S. Lange; S. Gebert; J. Spoerhase; P. Rygielski; T. Zinner; S. Kounev; P. Tran-Gia; in 27th International Teletraffic Congress (ITC 2015}) (2015). 210–218.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The Software Defined Networking~(SDN) concept introduces a paradigm shift in the networking world towards an externalized control plane which is logically centralized. When designing an SDN-based WAN architecture, it is of vital importance to find a feasible solution to the controller placement problem, i.e., to decide where to position a limited amount of resources within the network. In addition to time-independent constraints regarding aspects like scalability, resilience, and control plane communication delays, dynamically changing network conditions like traffic patterns or bandwidth demands need to be considered as well. Consequently, such dynamic environments call for a regular and fast recalculation of placements in order to adapt to the current situation in a timely manner. While an exhaustive evaluation of all possible solutions can be performed within a practically feasible time frame for small and medium-sized networks, such an approach is out of scope for large problem instances which have significantly higher time and memory requirements. Therefore, this work investigates a specialized heuristic, which takes into account a particular set of optimization objectives and returns solutions representing the possible trade-offs between them. Due to its low computation time and acceptable margin of error, this heuristic can be employed by automatic decision systems operating in dynamic environments.

  @inproceedings{LaGeSpRyZiKoTG-SDN-Controller-Placement-ITC2015, abstract = {{The Software Defined Networking (SDN) concept introduces a paradigm shift in the networking world towards an externalized control plane which is logically centralized. When designing an SDN-based WAN architecture, it is of vital importance to find a feasible solution to the controller placement problem, i.e., to decide where to position a limited amount of resources within the network. In addition to time-independent constraints regarding aspects like scalability, resilience, and control plane communication delays, dynamically changing network conditions like traffic patterns or bandwidth demands need to be considered as well. Consequently, such dynamic environments call for a regular and fast recalculation of placements in order to adapt to the current situation in a timely manner. While an exhaustive evaluation of all possible solutions can be performed within a practically feasible time frame for small and medium-sized networks, such an approach is out of scope for large problem instances which have significantly higher time and memory requirements. Therefore, this work investigates a specialized heuristic, which takes into account a particular set of optimization objectives and returns solutions representing the possible trade-offs between them. Due to its low computation time and acceptable margin of error, this heuristic can be employed by automatic decision systems operating in dynamic environments.}}, address = {Ghent, Belgium}, author = {Lange, Stanislav and Gebert, Steffen and Spoerhase, Joachim and Rygielski, Piotr and Zinner, Thomas and Kounev, Samuel and Tran-Gia, Phuoc}, booktitle = {27th International Teletraffic Congress ({ITC 2015})}, keywords = {Optimization}, month = {09}, pages = {210–218}, title = {{Specialized Heuristics for the Controller Placement Problem in Large Scale SDN Networks}}, year = 2015 }
• 
  Performance Isolation in Multi-Tenant Applications. R. Krebs; Thesis; Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany. (2015, August).
  • [ BibTeX ]
  @phdthesis{Krebs2015-phd, author = {Krebs, Rouven}, keywords = {descartes}, month = {08}, school = {Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany}, title = {Performance Isolation in Multi-Tenant Applications}, year = 2015 }
• 
  TableVisor: An Emulation Layer for Multi-Table OpenFlow Switches. S. Gebert; M. Jarschel; S. Herrnleben; T. Zinner; P. Tran-Gia; in 4th European Workshop on Software Defined Networks (EWSDN) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  This demonstration introduces TableVisor, which acts as a proxy layer between an OpenFlow controller and switches. Multiple hardware switches connect to TableVisor before it establishes a connection with the OpenFlow controller instance. During connection establishment, TableVisor advertises in the answer to the controller's features-request that this switch comprises a certain number of tables, which actually reflects the number of connected switches. All communication between controller and switch is passed through the proxy layer. Depending on the type of message, it will be answered directly by TableVisor (hello, features-reply), modified in such a way that particular fields are rewritten (packet-in, flow-mod), or a response merging data from multiple switches will be returned (flow-stats). The demonstration shows TableVisor in a MPLS use case using multiple switches, which is motivated by a mobile network scenario.

  @inproceedings{info3-inproceedings-2015-527, abstract = {This demonstration introduces TableVisor, which acts as a proxy layer between an OpenFlow controller and switches. Multiple hardware switches connect to TableVisor before it establishes a connection with the OpenFlow controller instance. During connection establishment, TableVisor advertises in the answer to the controller's features-request that this switch comprises a certain number of tables, which actually reflects the number of connected switches. All communication between controller and switch is passed through the proxy layer. Depending on the type of message, it will be answered directly by TableVisor (hello, features-reply), modified in such a way that particular fields are rewritten (packet-in, flow-mod), or a response merging data from multiple switches will be returned (flow-stats). The demonstration shows TableVisor in a MPLS use case using multiple switches, which is motivated by a mobile network scenario.}, address = {Bilbao, Spain}, author = {Gebert, Steffen and Jarschel, Michael and Herrnleben, Stefan and Zinner, Thomas and Tran-Gia, Phuoc}, booktitle = {4th European Workshop on Software Defined Networks (EWSDN)}, keywords = {t_workshop}, month = {09}, title = {TableVisor: An Emulation Layer for Multi-Table OpenFlow Switches}, year = 2015 }
• 
  Ingenieurmässige Entwicklung verlässlicher und effizienter Software-Systeme. S. Kounev; (2015, July).
  • [ BibTeX ]
  @misc{talk-Kounev-InvStutt15, author = {Kounev, Samuel}, howpublished = {Invited Talk, Faculty of Computer Science, Electrical Engineering and Information Technology, University of Stuttgart, Stuttgart, Germany}, keywords = {Talks}, month = {07}, title = {{Ingenieurm\"{a}ssige Entwicklung verl\"{a}sslicher und effizienter Software-Systeme}}, year = 2015 }
• 
  Proactive Memory Scaling of Virtualized Applications. S. Spinner; X. Zhu; L. Lu; M. Uysal; R. Griffith; N. Herbst; S. Kounev; in VMware R&D Innovation Offsite (RADIO) (2015).
  • [ BibTeX ]
  @inproceedings{SpinnerZLUGHK-2015-RADIO-MemoryScaling, author = {Spinner, Simon and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Griffith, Rean and Herbst, Nikolas and Kounev, Samuel}, booktitle = {VMware R&D Innovation Offsite (RADIO)}, keywords = {descartes}, month = {07}, title = {{Proactive Memory Scaling of Virtualized Applications}}, year = 2015 }
• 
  Demonstrating the Prospects of Dynamic Application-Aware Networking in a Home Environment. F. Wamser; T. Zinner; L. Iffländer; P. Tran-Gia; in ACM SIGCOMM Computer Communication Review (2015). 44(4) 149–150.
  • [ BibTeX ]
  • [ Download ]
  @article{wamser2015demonstrating, author = {Wamser, Florian and Zinner, Thomas and Iffländer, Lukas and Tran-Gia, Phuoc}, journal = {ACM SIGCOMM Computer Communication Review}, keywords = {t_journalmagazine}, month = {08}, number = 4, pages = {149–150}, publisher = {ACM}, title = {Demonstrating the Prospects of Dynamic Application-Aware Networking in a Home Environment}, volume = 44, year = 2015 }
• 
  Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8--10, 2015 S. Kounev; (P. Lalanda; A. Diaconescu; L. Cherkasova; S. Kounev, eds.) (2015). IEEE Computer Society Conference Publishing Services (CPS), Los Alamitos, California.
  • [ BibTeX ]
  @proceedings{proceedings-icac-2015, address = {Los Alamitos, California}, author = {Kounev, Samuel}, editor = {Lalanda, Philippe and Diaconescu, Ada and Cherkasova, Lucy and Kounev, Samuel}, keywords = {se2}, month = {07}, publisher = {IEEE Computer Society Conference Publishing Services (CPS)}, title = {Proceeding of IEEE International Conference on Autonomic Computing (ICAC2015), Grenoble, France, July 8–10, 2015}, year = 2015 }
• 
  Flexible Performance Prediction of Data Center Networks using Automatically Generated Simulation Models. P. Rygielski; S. Kounev; P. Tran-Gia; in Proceedings of the Eighth International Conference on Simulation Tools and Techniques (SIMUTools 2015) (2015). 119–128.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Using different modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. In this paper, we propose a generic approach to modeling the performance of data center networks. The approach offers multiple performance models but requires to use only a single modeling language. We propose a two-step modeling methodology, in which a high-level descriptive model of the network is built in the first step, and in the second step model-to-model transformations are used to automatically transform the descriptive model to different network simulation models. We automatically generate three performance models defined at different levels of abstraction to analyze network throughput. By offering multiple simulation models in parallel, we provide flexibility in trading-off between the modeling accuracy and the simulation overhead. We analyze the simulation models by comparing the prediction accuracy with respect to the simulation duration. We observe, that in the investigated scenarios the solution duration of coarser simulation models is up to 300 times shorter, whereas the average prediction accuracy decreases only by 4 percent.

  @inproceedings{RyKoTr2015-SIMUTools-FlexPerformancePrediction, abstract = {{Using different modeling and simulation approaches for predicting network performance requires extensive experience and involves a number of time consuming manual steps regarding each of the modeling formalisms. In this paper, we propose a generic approach to modeling the performance of data center networks. The approach offers multiple performance models but requires to use only a single modeling language. We propose a two-step modeling methodology, in which a high-level descriptive model of the network is built in the first step, and in the second step model-to-model transformations are used to automatically transform the descriptive model to different network simulation models. We automatically generate three performance models defined at different levels of abstraction to analyze network throughput. By offering multiple simulation models in parallel, we provide flexibility in trading-off between the modeling accuracy and the simulation overhead. We analyze the simulation models by comparing the prediction accuracy with respect to the simulation duration. We observe, that in the investigated scenarios the solution duration of coarser simulation models is up to 300 times shorter, whereas the average prediction accuracy decreases only by 4 percent.}}, author = {Rygielski, Piotr and Kounev, Samuel and Tran-Gia, Phuoc}, booktitle = {Proceedings of the Eighth International Conference on Simulation Tools and Techniques (SIMUTools 2015)}, keywords = {QPN}, month = {08}, pages = {119–128}, title = {{Flexible Performance Prediction of Data Center Networks using Automatically Generated Simulation Models}}, year = 2015 }
• 
  Parallel Simulation of Queueing Petri Nets. J. Walter; S. Spinner; S. Kounev; in Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015) (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

  @inproceedings{WaSpKo-SIMUTools-QPNParallelSimulation, abstract = {Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.}, author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel}, booktitle = {Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015)}, keywords = {QPN}, month = {08}, title = {{Parallel Simulation of Queueing Petri Nets}}, year = 2015 }
• 
  Parallel Simulation of Queueing Petri Nets. J. Walter; S. Spinner; S. Kounev; in EAI Endorsed Transactions on Industrial Networks and Intelligent Systems (2015). 16(8)
  • [ BibTeX ]
  @article{WaSpKo2016-EAI-parallelSimulation, author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel}, journal = {EAI Endorsed Transactions on Industrial Networks and Intelligent Systems}, keywords = {se2}, month = {08}, number = 8, publisher = {ACM}, title = {{Parallel Simulation of Queueing Petri Nets}}, volume = 16, year = 2015 }
• 
  Performance-oriented DevOps: A Research Agenda. (SPEC-RG-2015-01), A. Brunnert; A. van Hoorn; F. Willnecker; A. Danciu; W. Hasselbring; C. Heger; N. Herbst; P. Jamshidi; R. Jung; J. von Kistowski; A. Koziolek; J. Kross; S. Spinner; C. Vögele; J. Walter; A. Wert; (2015).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.

  @techreport{SPEC-RG-2015-001-DevOpsPerformanceResearchAgenda, abstract = {DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.}, author = {Brunnert, Andreas and van Hoorn, Andre and Willnecker, Felix and Danciu, Alexandru and Hasselbring, Wilhelm and Heger, Christoph and Herbst, Nikolas and Jamshidi, Pooyan and Jung, Reiner and von Kistowski, Joakim and Koziolek, Anne and Kross, Johannes and Spinner, Simon and V{\"o}gele, Christian and Walter, J{\"u}rgen and Wert, Alexander}, institution = {SPEC Research Group — DevOps Performance Working Group, Standard Performance Evaluation Corporation (SPEC)}, keywords = {SPEC}, month = {08}, number = {SPEC-RG-2015-01}, title = {Performance-oriented {DevOps}: {A} Research Agenda}, year = 2015 }

• 
  Data Center Network Throughput Analysis using Queueing Petri Nets. P. Rygielski; S. Kounev; in 34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, (DCPerf 2014) (2014). 100–105.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper, we address the problem of performance analysis in computer networks. We present a metamodel designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. In this paper, we present the transformation to Queueing Petri Nets (QPNs) and simulate them using SimQPN simulator. We show that despite the high level of abstraction of the QPNs, we are able to estimate the utilization of resources with good accuracy within a short time.

  @inproceedings{RyKo2014-DCPerf-DNI2QPN, abstract = {In this paper, we address the problem of performance analysis in computer networks. We present a metamodel designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. In this paper, we present the transformation to Queueing Petri Nets (QPNs) and simulate them using SimQPN simulator. We show that despite the high level of abstraction of the QPNs, we are able to estimate the utilization of resources with good accuracy within a short time.}, author = {Rygielski, Piotr and Kounev, Samuel}, booktitle = {34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, (DCPerf 2014)}, keywords = {DNI}, month = {06}, pages = {100–105}, title = {{Data Center Network Throughput Analysis using Queueing Petri Nets}}, year = 2014 }
• 
  Towards a Resource Elasticity Benchmark for Cloud Environments. A. Weber; N. R. Herbst; H. Groenda; S. Kounev; in Proceedings of the 2nd International Workshop on Hot Topics in Cloud Service Scalability (HotTopiCS 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). 5:1–5:8.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Auto-scaling features offered by today's cloud infrastructures provide increased flexibility especially for customers that experience high variations in the load intensity over time. However, auto-scaling features introduce new system quality attributes when considering their accuracy, timing, and boundaries. Therefore, distinguishing between different offerings has become a complex task, as it is not yet supported by reliable metrics and measurement approaches. In this paper, we discuss shortcomings of existing approaches for measuring and evaluating elastic behavior and propose a novel benchmark methodology specifically designed for evaluating the elasticity aspects of modern cloud platforms. The benchmark is based on open workloads with realistic load variation profiles that are calibrated to induce identical resource demand variations independent of the underlying hardware performance. Furthermore, we propose new metrics that capture the accuracy of resource allocations and de-allocations, as well as the timing aspects of an auto-scaling mechanism explicitly.

  @inproceedings{WeHeGrKo2014-HotTopicsWS-ElaBench, abstract = {{Auto-scaling features offered by today's cloud infrastructures provide increased flexibility especially for customers that experience high variations in the load intensity over time. However, auto-scaling features introduce new system quality attributes when considering their accuracy, timing, and boundaries. Therefore, distinguishing between different offerings has become a complex task, as it is not yet supported by reliable metrics and measurement approaches. In this paper, we discuss shortcomings of existing approaches for measuring and evaluating elastic behavior and propose a novel benchmark methodology specifically designed for evaluating the elasticity aspects of modern cloud platforms. The benchmark is based on open workloads with realistic load variation profiles that are calibrated to induce identical resource demand variations independent of the underlying hardware performance. Furthermore, we propose new metrics that capture the accuracy of resource allocations and de-allocations, as well as the timing aspects of an auto-scaling mechanism explicitly.}}, address = {New York, NY, USA}, author = {Weber, Andreas and Herbst, Nikolas Roman and Groenda, Henning and Kounev, Samuel}, booktitle = {Proceedings of the 2nd International Workshop on Hot Topics in Cloud Service Scalability (HotTopiCS 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {t_workshop}, month = {03}, pages = {5:1–5:8}, publisher = {ACM}, series = {HotTopiCS '14}, title = {{Towards a Resource Elasticity Benchmark for Cloud Environments}}, year = 2014 }
• 
  libieeep1788: A C++ Implementation of the IEEE interval standard P1788. M. Nehmeier; in Norbert Wiener in the 21st Century (21CW), 2014 IEEE Conference on (2014). 1–6.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In 2008 the IEEE interval standard working group P1788 has been founded to enlarge the acceptance of interval arithmetic. One of the main challenges for the working group was to cope with different approaches of interval arithmetic. Such as set based interval arithmetic or modal interval arithmetic, which have been developed during the last 50 years. For this reason a concept called flavors is defined by the standard. The concept specifies the behavior of the operations. Additionally, a so called decoration system is introduced to treat mathematical events like discontinuity or undefinedness of an expression for a given interval box. The C++ library libieeep1788 tries to implement all features of the tentative interval standard in a clear and straightforward manner. The main goal is to have a faithful reference implementation of the standard and not to use dubious optimizations and hacks. The main design concept of the library is to map the flavors concept one-to-one onto the implementation. To achieve this requirement the so called policy based class design is used. In this paper we will present the key concepts of the faithful C++ library libieeep1788 which will be accompanied by an overview of the preliminary IEEE P1788 interval standard.

  @inproceedings{nehmeierWiener2014, abstract = {In 2008 the IEEE interval standard working group P1788 has been founded to enlarge the acceptance of interval arithmetic. One of the main challenges for the working group was to cope with different approaches of interval arithmetic. Such as set based interval arithmetic or modal interval arithmetic, which have been developed during the last 50 years. For this reason a concept called flavors is defined by the standard. The concept specifies the behavior of the operations. Additionally, a so called decoration system is introduced to treat mathematical events like discontinuity or undefinedness of an expression for a given interval box. The C++ library libieeep1788 tries to implement all features of the tentative interval standard in a clear and straightforward manner. The main goal is to have a faithful reference implementation of the standard and not to use dubious optimizations and hacks. The main design concept of the library is to map the flavors concept one-to-one onto the implementation. To achieve this requirement the so called policy based class design is used. In this paper we will present the key concepts of the faithful C++ library libieeep1788 which will be accompanied by an overview of the preliminary IEEE P1788 interval standard.}, author = {Nehmeier, Marco}, booktitle = {Norbert Wiener in the 21st Century (21CW), 2014 IEEE Conference on}, keywords = {Reliability}, month = {06}, pages = {1-6}, title = {libieeep1788: {A} {C++} {I}mplementation of the {IEEE} interval standard {P1788}}, year = 2014 }
• 
  Enriching Software Architecture Models with Statistical Models for Performance Prediction in Modern Storage Environments. Q. Noorshams; R. Reeb; A. Rentschler; S. Kounev; R. Reussner; in Proceedings of the 17th International ACM Sigsoft Symposium on Component-based Software Engineering (2014). 45–54.

  Acceptance Rate (Full Paper): 14/62 = 23%.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoReReKoRe2014-CBSE-ArchitectureModelsWithStatisticalModels, address = {New York, NY, USA}, author = {Noorshams, Qais and Reeb, Roland and Rentschler, Andreas and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the 17th International ACM Sigsoft Symposium on Component-based Software Engineering}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = {06}, note = {Acceptance Rate (Full Paper): 14/62 = 23\%.}, pages = {45–54}, publisher = {ACM}, series = {CBSE '14}, title = {Enriching Software Architecture Models with Statistical Models for Performance Prediction in Modern Storage Environments}, year = 2014 }
• 
  Ingenieurmässige Entwicklung verlässlicher und effizienter Software-Systeme. S. Kounev; (2014, May).
  • [ BibTeX ]
  @misc{talk-Kounev-WelcomeWue14, author = {Kounev, Samuel}, howpublished = {Faculty of Mathematics and Computer Science, University of W{\"u}rzburg}, keywords = {Talks}, month = {05}, title = {{Ingenieurm\"{a}ssige Entwicklung verl\"{a}sslicher und effizienter Software-Systeme}}, year = 2014 }
• 
  Resource Usage Control In Multi-Tenant Applications. R. Krebs; S. Spinner; N. Ahmed; S. Kounev; in Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014) (2014). 122–131.

  Acceptance Rate: 19% (54/283)
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.

  @inproceedings{KrSpAhKo2014_CCGrid_ResourceIsolation, abstract = {{Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by today's platform environments.}}, author = {Krebs, Rouven and Spinner, Simon and Ahmed, Nadia and Kounev, Samuel}, booktitle = {Proceedings of the 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid 2014)}, keywords = {Self-adaptive-systems}, month = {05}, note = {Acceptance Rate: 19\% (54/283)}, pages = {122-131}, publisher = {IEEE/ACM}, title = {{Resource Usage Control In Multi-Tenant Applications}}, year = 2014 }
• 
  Locations for Performance Ensuring Admission Control in Load Balanced Multi-Tenant Systems. M. Loesch; R. Krebs; in Communications in Computer and Information Science - Third International Conference, CLOSER 2013, Aachen, Germany, May 8-10, 2013, Selected Papers, Springer (ed.) (2014).
  • [ Abstract ]
  • [ BibTeX ]
  In the context of Software as a Service offerings, multi-tenant applications (MTAs) allow to increase the efficiency by sharing one application instance among several customers. Due to the tight coupling and sharing of resources on all layers up to the application layer, customers may influence each other with regards to the performance they observe. Existing research on performance isolation of MTAs focuses on methods and concrete algorithms. In this paper, we present concerns that are raised when serving a high amount of users in a load balancing cluster with multiple MTA instances. We identified potential positions in such an architecture where performance isolation can be enforced based on request admission control. Considering various approaches for request-to-instance allocation, our discussion shows that different positions come along with specific pros and cons that have influence on the ability to performance-isolate tenants.

  @inproceedings{LoKr2014-CCIS-LocationsForIsolation, abstract = {In the context of Software as a Service offerings, multi-tenant applications (MTAs) allow to increase the efficiency by sharing one application instance among several customers. Due to the tight coupling and sharing of resources on all layers up to the application layer, customers may influence each other with regards to the performance they observe. Existing research on performance isolation of MTAs focuses on methods and concrete algorithms. In this paper, we present concerns that are raised when serving a high amount of users in a load balancing cluster with multiple MTA instances. We identified potential positions in such an architecture where performance isolation can be enforced based on request admission control. Considering various approaches for request-to-instance allocation, our discussion shows that different positions come along with specific pros and cons that have influence on the ability to performance-isolate tenants.}, author = {Loesch, Manuel and Krebs, Rouven}, booktitle = {Communications in Computer and Information Science - Third International Conference, CLOSER 2013, Aachen, Germany, May 8-10, 2013, Selected Papers}, editor = {Springer}, keywords = {se2}, month = {05}, title = {{Locations for Performance Ensuring Admission Control in Load Balanced Multi-Tenant Systems}}, year = 2014 }
• 
  Designing Information Hiding Modularity for Model Transformation Languages. A. Rentschler; D. Werle; Q. Noorshams; L. Happe; R. Reussner; in Proceedings of the 13th International Conference on Modularity (AOSD ’14), Lugano, Switzerland, April 22 - 26, 2014 (2014). 217–228.

  Acceptance Rate: 35.0%
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{ReWeNoHaRe2014-AOSD-Modularity, address = {New York, NY, USA}, author = {Rentschler, Andreas and Werle, Dominik and Noorshams, Qais and Happe, Lucia and Reussner, Ralf}, booktitle = {Proceedings of the 13th International Conference on Modularity (AOSD '14), Lugano, Switzerland, April 22 - 26, 2014}, keywords = {se2}, month = {04}, note = {Acceptance Rate: 35.0\%}, pages = {217–228}, publisher = {ACM}, title = {{Designing Information Hiding Modularity for Model Transformation Languages}}, year = 2014 }
• 
  Comparison of Request Admission Based Performance Isolation Approaches in Multi-Tenant SaaS Applications (Short Paper). R. Krebs; M. Loesch; in Proceedings of 4th International Conference On Cloud Computing And Services Science (CLOSER 2014) (2014).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by todays platform environments.

  @inproceedings{KrLo2014_Closer_IsolationTypes, abstract = {{Multi-tenancy is an approach to share one application instance among multiple customers by providing each of them a dedicated view. This approach is commonly used by SaaS providers to reduce the costs for service provisioning. Tenants also expect to be isolated in terms of the performance they observe and the providers inability to offer performance guarantees is a major obstacle for potential cloud customers. To guarantee an isolated performance it is essential to control the resources used by a tenant. This is a challenge, because the layers of the execution environment, responsible for controlling resource usage (e.g., operating system), normally do not have knowledge about entities defined at the application level and thus they cannot distinguish between different tenants. Furthermore, it is hard to predict how tenant requests propagate through the multiple layers of the execution environment down to the physical resource layer. The intended abstraction of the application from the resource controlling layers does not allow to solely solving this problem in the application. In this paper, we propose an approach which applies resource demand estimation techniques in combination with a request based admission control. The resource demand estimation is used to determine resource consumption information for individual requests. The admission control mechanism uses this knowledge to delay requests originating from tenants that exceed their allocated resource share. The proposed method is validated by a widely accepted benchmark showing its applicability in a setup motivated by todays platform environments.}}, author = {Krebs, Rouven and Loesch, Manuel}, booktitle = {Proceedings of 4th International Conference On Cloud Computing And Services Science (CLOSER 2014)}, keywords = {se2}, month = {04}, publisher = {SciTePress}, title = {{Comparison of Request Admission Based Performance Isolation Approaches in Multi-Tenant SaaS Applications (Short Paper)}}, year = 2014 }
• 
  SPEC Research Group Newsletter, vol. 1 no. 3. S. Kounev; K. Sachs; P. Rygielski; (2014, March).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-3, author = {Kounev, Samuel and Sachs, Kai and Rygielski, Piotr}, keywords = {SPEC}, month = {03}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 1 no. 3}}, year = 2014 }
• 
  Optimization Method for Request Admission Control to Guarantee Performance Isolation. R. Krebs; P. Schneider; N. Herbst; in Proceedings of the 2nd International Workshop on Hot Topics in Cloud Service Scalability (HotTopiCS 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Software-as-a-Service (SaaS) often shares one single application instance among different tenants to reduce costs. However, sharing potentially leads to undesired influence from one tenant onto the performance observed by the others. Furthermore, providing one tenant additional resources to support its increasing demands without increasing the performance of tenants who do not pay for it is a major challenge. The application intentionally does not manage hardware resources, and the OS is not aware of application level entities like tenants. Thus, it is difficult to control the performance of different tenants to keep them isolated. These problems gain importance as performance is one of the major obstacles for cloud customers. Existing work applies request based admission control mechanisms like a weighted round robin with an individual queue for each tenant to control the share guaranteed for a tenant. However, the computation of the concrete weights for such an admission control is still challenging. In this paper, we present a fitness function and optimization approach reflecting various requirements from this field to compute proper weights with the goal to ensure an isolated performance as foundation to scale on a tenants basis.

  @inproceedings{KrScHe2014-HotTopiCS-OptimizationApproach, abstract = {{Software-as-a-Service (SaaS) often shares one single application instance among different tenants to reduce costs. However, sharing potentially leads to undesired influence from one tenant onto the performance observed by the others. Furthermore, providing one tenant additional resources to support its increasing demands without increasing the performance of tenants who do not pay for it is a major challenge. The application intentionally does not manage hardware resources, and the OS is not aware of application level entities like tenants. Thus, it is difficult to control the performance of different tenants to keep them isolated. These problems gain importance as performance is one of the major obstacles for cloud customers. Existing work applies request based admission control mechanisms like a weighted round robin with an individual queue for each tenant to control the share guaranteed for a tenant. However, the computation of the concrete weights for such an admission control is still challenging. In this paper, we present a fitness function and optimization approach reflecting various requirements from this field to compute proper weights with the goal to ensure an isolated performance as foundation to scale on a tenants basis.}}, author = {Krebs, Rouven and Schneider, Philipp and Herbst, Nikolas}, booktitle = {Proceedings of the 2nd International Workshop on Hot Topics in Cloud Service Scalability (HotTopiCS 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {Optimization}, month = {03}, publisher = {ACM}, title = {{Optimization Method for Request Admission Control to Guarantee Performance Isolation}}, year = 2014 }
• 
  Modeling Variations in Load Intensity over Time. J. von Kistowski; N. R. Herbst; S. Kounev; in Proceedings of the 3rd International Workshop on Large-Scale Testing (LT 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). 1–4.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Today's software systems are expected to deliver reliable performance under highly variable load intensities while at the same time making efficient use of dynamically allocated resources. Conventional benchmarking frameworks provide limited support for emulating such highly variable and dynamic load profiles and workload scenarios. Industrial benchmarks typically use workloads with constant or stepwise increasing load intensity, or they simply replay recorded workload traces. Based on this observation, we identify the need for means allowing flexible definition of load profiles and address this by introducing two meta-models at different abstraction levels. At the lower abstraction level, the Descartes Load Intensity Meta-Model (DLIM) offers a structured and accessible way of describing the load intensity over time by editing and combining mathematical functions. The High-Level Descartes Load Intensity Meta-Model (HLDLIM) allows the description of load variations using few defined parameters that characterize the seasonal patterns, trends, bursts and noise parts. We demonstrate that both meta-models are capable of capturing real-world load profiles with acceptable accuracy through comparison with a real life trace.

  @inproceedings{KiHeKo2014-LT-DLIM, abstract = {{Today's software systems are expected to deliver reliable performance under highly variable load intensities while at the same time making efficient use of dynamically allocated resources. Conventional benchmarking frameworks provide limited support for emulating such highly variable and dynamic load profiles and workload scenarios. Industrial benchmarks typically use workloads with constant or stepwise increasing load intensity, or they simply replay recorded workload traces. Based on this observation, we identify the need for means allowing flexible definition of load profiles and address this by introducing two meta-models at different abstraction levels. At the lower abstraction level, the Descartes Load Intensity Meta-Model (DLIM) offers a structured and accessible way of describing the load intensity over time by editing and combining mathematical functions. The High-Level Descartes Load Intensity Meta-Model (HLDLIM) allows the description of load variations using few defined parameters that characterize the seasonal patterns, trends, bursts and noise parts. We demonstrate that both meta-models are capable of capturing real-world load profiles with acceptable accuracy through comparison with a real life trace.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas Roman and Kounev, Samuel}, booktitle = {Proceedings of the 3rd International Workshop on Large-Scale Testing (LT 2014), co-located with the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {t_workshop}, month = {03}, pages = {1–4}, publisher = {ACM}, title = {{Modeling Variations in Load Intensity over Time}}, year = 2014 }
• 
  LIMBO - A Load Intensity Modeling Platform. J. von Kistowski; (2014, March).
  • [ BibTeX ]
  @misc{talk-Kistowski-SPECRG2014, author = {von Kistowski, J{\'o}akim}, howpublished = {Talk at SPEC RG Annual Meeting 2014}, keywords = {LIMBO}, month = {03}, title = {{LIMBO - A Load Intensity Modeling Platform}}, year = 2014 }
• 
  LibReDE: A Library for Resource Demand Estimation. S. Spinner; G. Casale; X. Zhu; S. Kounev; in Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). 227–228.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  When creating a performance model, it is necessary to quantify the amount of resources consumed by an application serving individual requests. In distributed enterprise systems, these resource demands usually cannot be observed directly, their estimation is a major challenge. Different statistical approaches to resource demand estimation based on monitoring data have been proposed, e.g., using linear regression or Kalman filtering techniques. In this paper, we present LibReDE, a library of ready-to-use implementations of approaches to resource demand estimation that can be used for online and offline analysis. It is the first publicly available tool for this task and aims at supporting performance engineers during performance model construction. The library enables the quick comparison of the estimation accuracy of different approaches in a given context and thus helps to select an optimal one.

  @inproceedings{SpCaZhKo2014-ICPEDemo-LibReDE, abstract = {When creating a performance model, it is necessary to quantify the amount of resources consumed by an application serving individual requests. In distributed enterprise systems, these resource demands usually cannot be observed directly, their estimation is a major challenge. Different statistical approaches to resource demand estimation based on monitoring data have been proposed, e.g., using linear regression or Kalman filtering techniques. In this paper, we present LibReDE, a library of ready-to-use implementations of approaches to resource demand estimation that can be used for online and offline analysis. It is the first publicly available tool for this task and aims at supporting performance engineers during performance model construction. The library enables the quick comparison of the estimation accuracy of different approaches in a given context and thus helps to select an optimal one.}, address = {New York, NY, USA}, author = {Spinner, Simon and Casale, Giuliano and Zhu, Xiaoyun and Kounev, Samuel}, booktitle = {Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = {03}, pages = {227–228}, publisher = {ACM Press}, title = {{LibReDE: A Library for Resource Demand Estimation}}, year = 2014 }
• 
  Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning. N. R. Herbst; N. Huber; S. Kounev; E. Amrehn; in Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd. (2014). 26(12) 2053–2078.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.

  @article{HuKoAm2013-CCPE-WorkloadClassificationAndForecasting, abstract = {As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.}, author = {Herbst, Nikolas Roman and Huber, Nikolaus and Kounev, Samuel and Amrehn, Erich}, journal = {Concurrency and Computation - Practice and Experience, John Wiley and Sons, Ltd.}, keywords = {Self-adaptive-systems}, month = {03}, number = 12, pages = {2053–2078}, title = {{Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning}}, volume = 26, year = 2014 }
• 
  Constructing Performance Model of JMS Middleware Platform. T. Martinec; L. Marek; A. Steinhauser; P. Tuma; Q. Noorshams; A. Rentschler; R. Reussner; in Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (2014). 123–134.
  • [ BibTeX ]
  • [ URL ]
  @inproceedings{MaMaStTuNoReRe2014-ICPE-JMSModel, address = {New York, NY, USA}, author = {Martinec, Tom{\'a}c{s} and Marek, Luk{\'a}\c{s} and Steinhauser, Anton{\'i}n and T{u}ma, Petr and Noorshams, Qais and Rentschler, Andreas and Reussner, Ralf}, booktitle = {Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering}, keywords = {se2}, month = {03}, pages = {123–134}, publisher = {ACM}, series = {ICPE '14}, title = {Constructing Performance Model of JMS Middleware Platform}, year = 2014 }
• 
  LIMBO: A Tool For Modeling Variable Load Intensities. J. von Kistowski; N. R. Herbst; S. Kounev; in Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014). 225–226.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modern software systems are expected to deliver reliable performance under highly variable load intensities while at the same time making efficient use of dynamically allocated resources. Conventional benchmarking frameworks provide limited support for emulating such highly variable and dynamic load profiles and workload scenarios. Industrial benchmarks typically use workloads with constant or stepwise increasing load intensity, or they simply replay recorded workload traces. In this paper, we present LIMBO - an Eclipse-based tool for modeling variable load intensity profiles based on the Descartes Load Intensity Model as an underlying modeling formalism.

  @inproceedings{KiHeKo2014-ICPEDemo-LIMBO, abstract = {{Modern software systems are expected to deliver reliable performance under highly variable load intensities while at the same time making efficient use of dynamically allocated resources. Conventional benchmarking frameworks provide limited support for emulating such highly variable and dynamic load profiles and workload scenarios. Industrial benchmarks typically use workloads with constant or stepwise increasing load intensity, or they simply replay recorded workload traces. In this paper, we present LIMBO - an Eclipse-based tool for modeling variable load intensity profiles based on the Descartes Load Intensity Model as an underlying modeling formalism.}}, address = {New York, NY, USA}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas Roman and Kounev, Samuel}, booktitle = {Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {LIMBO}, month = {03}, pages = {225–226}, publisher = {ACM}, series = {ICPE '14}, title = {{LIMBO: A Tool For Modeling Variable Load Intensities}}, year = 2014 }
• 
  Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments. R. Krebs; C. Momm; S. Kounev; in Elsevier Science of Computer Programming Journal (SciCo) (2014). Vol. 90, Part B 116–134.
  • [ BibTeX ]
  • [ Download ]
  @article{KrMoKo2013-SciCo-MetricsAndTechniquesForPerformanceIsolation, author = {Krebs, Rouven and Momm, Christof and Kounev, Samuel}, journal = {Elsevier Science of Computer Programming Journal (SciCo)}, keywords = {Virtualization}, month = {02}, pages = {116–134}, publisher = {Elsevier B.V.}, title = {{Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments}}, volume = {Vol. 90, Part B}, year = 2014 }
• 
  Autonomic Performance-Aware Resource Management in Dynamic IT Service Infrastructures. N. Huber; Thesis; Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany. (2014, July).
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Huber2014-KIT-AutonomicPerformanceAwareResourceManagement, author = {Huber, Nikolaus}, keywords = {Self-adaptive-systems}, month = {07}, school = {Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany}, title = {Autonomic Performance-Aware Resource Management in Dynamic IT Service Infrastructures}, year = 2014 }
• 
  Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems. Q. Noorshams; A. Busch; A. Rentschler; D. Bruhn; S. Kounev; P. Tuma; R. Reussner; in 34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, DCPerf ’14 (2014). 88–93.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoBuReBrKoTuRe2014-DCPerf-IORegressionModels, author = {Noorshams, Qais and Busch, Axel and Rentschler, Andreas and Bruhn, Dominik and Kounev, Samuel and T{u}ma, Petr and Reussner, Ralf}, booktitle = {34th IEEE International Conference on Distributed Computing Systems Workshops (ICDCS 2014 Workshops). 4th International Workshop on Data Center Performance, DCPerf '14}, keywords = {t_workshop}, month = {07}, pages = {88-93}, title = {{Automated Modeling of I/O Performance and Interference Effects in Virtualized Storage Systems}}, year = 2014 }
• 
  Modeling of I/O Performance Interference in Virtualized Environments with Queueing Petri Nets. Q. Noorshams; K. Rostami; S. Kounev; R. Reussner; in Proceedings of the IEEE 22nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (2014).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{NoRoKoRe2014-MASCOTS-IOQPNs, author = {Noorshams, Qais and Rostami, Kiana and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the IEEE 22nd International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems}, keywords = {Virtualization}, month = {{September}}, series = {MASCOTS 2014}, title = {{Modeling of I/O Performance Interference in Virtualized Environments with Queueing Petri Nets}}, year = 2014 }
• 
  LibReDE - A library for Resource Demand Estimation. S. Spinner; (2014, July).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Spinner-SPECdevops14, author = {Spinner, Simon}, howpublished = {Technical Talk at SPEC DevOps Performance Working Group}, keywords = {descartes}, month = {07}, title = {{LibReDE - A library for Resource Demand Estimation}}, year = 2014 }
• 
  The Descartes Modeling Language S. Kounev; F. Brosig; N. Huber; (2014).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This technical report introduces the Descartes Modeling Language (DML), a new architecture-level modeling language for modeling Quality-of-Service (QoS) and resource management related aspects of modern dynamic IT systems, infrastructures and services. DML is designed to serve as a basis for self-aware resource management during operation ensuring that system QoS requirements are continuously satisfied while infrastructure resources are utilized as efficiently as possible.

  @techreport{KoBrHu2014-TechReport-DML, abstract = {{This technical report introduces the Descartes Modeling Language (DML), a new architecture-level modeling language for modeling Quality-of-Service (QoS) and resource management related aspects of modern dynamic IT systems, infrastructures and services. DML is designed to serve as a basis for self-aware resource management during operation ensuring that system QoS requirements are continuously satisfied while infrastructure resources are utilized as efficiently as possible.}}, author = {Kounev, Samuel and Brosig, Fabian and Huber, Nikolaus}, institution = {{Department of Computer Science, University of Wuerzburg}}, keywords = {Self-adaptive-systems}, month = 10, title = {{The Descartes Modeling Language}}, year = 2014 }
• 
  Platform-as-a-Service Architecture for Performance Isolated Multi-Tenant Applications. R. Krebs; M. Loesch; S. Kounev; in Proceedings of the 7th IEEE International Conference on Cloud Computing (IEEE Cloud 2014) (2014).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Software-as-a-Service (SaaS) often shares one single application instance among different tenants to reduce costs. However, sharing potentially leads to undesired influence from one tenant onto the performance observed by the others. This is a significant problem as performance is one of the major obstacles for cloud customers. The application does intentionally not manage hardware resources, and the operating system is not aware of application level entities like tenants which makes the performance control a challenge. In case the SaaS is hosted on a Platform-as-a-Service (PaaS), the SaaS developer usually wants to control performance-related issues according to individual needs, and available information is even more limited. Thus, it is difficult to control the performance of different tenants to keep them isolated. Existing work focuses on concrete methods to provide performance isolation in systems where the whole stack is under control. In this paper we present a concrete PaaS enhancement which enables application developers to realize isolation methods for their hosted SaaS application. In a case study we evaluated the applicability and effectiveness of the enhancement in different environments.

  @inproceedings{KrLoKo2014_CLOUD_Framework, abstract = {{Software-as-a-Service (SaaS) often shares one single application instance among different tenants to reduce costs. However, sharing potentially leads to undesired influence from one tenant onto the performance observed by the others. This is a significant problem as performance is one of the major obstacles for cloud customers. The application does intentionally not manage hardware resources, and the operating system is not aware of application level entities like tenants which makes the performance control a challenge. In case the SaaS is hosted on a Platform-as-a-Service (PaaS), the SaaS developer usually wants to control performance-related issues according to individual needs, and available information is even more limited. Thus, it is difficult to control the performance of different tenants to keep them isolated. Existing work focuses on concrete methods to provide performance isolation in systems where the whole stack is under control. In this paper we present a concrete PaaS enhancement which enables application developers to realize isolation methods for their hosted SaaS application. In a case study we evaluated the applicability and effectiveness of the enhancement in different environments.}}, author = {Krebs, Rouven and Loesch, Manuel and Kounev, Samuel}, booktitle = {Proceedings of the 7th IEEE International Conference on Cloud Computing (IEEE Cloud 2014)}, keywords = {Application_quality_of_service_management}, month = {07}, publisher = {IEEE}, title = {{Platform-as-a-Service Architecture for Performance Isolated Multi-Tenant Applications}}, year = 2014 }
• 
  Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments. N. Huber; A. van Hoorn; A. Koziolek; F. Brosig; S. Kounev; in Service Oriented Computing and Applications Journal (SOCA) (2014). 8(1) 73–89.
  • [ BibTeX ]
  • [ Download ]
  @article{HuHoKoBrKo2014-SOCA-ModelingRuntimeAdaptation, author = {Huber, Nikolaus and van Hoorn, Andr{\'e} and Koziolek, Anne and Brosig, Fabian and Kounev, Samuel}, journal = {Service Oriented Computing and Applications Journal (SOCA)}, keywords = {Self-adaptive-systems}, month = {{March}}, number = 1, pages = {73–89}, publisher = {Springer-Verlag}, title = {{Modeling Run-Time Adaptation at the System Architecture Level in Dynamic Service-Oriented Environments}}, volume = 8, year = 2014 }
• 
  Performance Queries for Architecture-Level Performance Models. F. Gorsler; F. Brosig; S. Kounev; in Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014) (2014).

  Acceptance Rate (Full Paper): 14/56 = 25%.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{GoBrKo2014-PerformanceQueries, address = {New York, NY, USA}, author = {Gorsler, Fabian and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014)}, keywords = {DQL}, month = {{March}}, note = {Acceptance Rate (Full Paper): 14/56 = 25\%.}, publisher = {ACM}, title = {{Performance Queries for Architecture-Level Performance Models}}, year = 2014 }
• 
  Parallel Simulation of Queueing Petri Nets. J. Walter; S. Spinner; S. Kounev; (2014, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Walter-SOSP14, author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel}, howpublished = {Technical Talk at Symposion on Software Performance (SOSP) 2014}, keywords = {QPN}, month = 11, title = {{Parallel Simulation of Queueing Petri Nets}}, year = 2014 }
• 
  LibReDE: A Library for Resource Demand Estimation. S. Spinner; J. Walter; (2014, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Spinner-SOSP14, author = {Spinner, Simon and Walter, J{\"u}rgen}, howpublished = {Technical Talk at Symposion on Software Performance (SOSP) 2014}, keywords = {Talks}, month = 11, title = {{LibReDE: A Library for Resource Demand Estimation}}, year = 2014 }
• 
  The Descartes Modeling Language: Status Quo. S. Kounev; (2014, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-SOSP14, author = {Kounev, Samuel}, howpublished = {Technical Talk at the Symposium on Software Performance (SOSP 2014), Joint Descartes/Kieker/Palladio Days (Joint Developer and Community Meeting), Stuttgart, Germany}, keywords = {Talks}, month = 11, title = {{The Descartes Modeling Language: Status Quo}}, year = 2014 }
• 
  Using and Extending LIMBO for the Descriptive Modeling of Arrival Behaviors. J. von Kistowski; N. R. Herbst; S. Kounev; in Proceedings of the Symposium on Software Performance 2014 (2014). 131–140.

  Best Poster Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  LIMBO is a tool for the creation of load profiles with variable intensity over time both from scratch and from existing data. Primarily, LIMBO's intended use is the description of load arrival behaviours in open workloads. Specifically, LIMBO can be employed for the creation of custom request or user arrival time-stamps or for the re-parameterization of existing traces. LIMBO bases on the Descartes Load Intensity Model (DLIM) for the formalized description of its load intensity profiles. The DLIM formalism can be understood as a structure for piece-wise defined and combined mathematical functions. We outline DLIM and its elements and demonstrate its integration within LIMBO. LIMBO is capable of generating request or user arrival time stamps from DLIM instances. In a next step, these generated time-stamps can be used for both open workload based benchmarking and simulations. The TimestampTimer plug-in for JMeter already allows the former. LIMBO also offers a range of tools for easy load intensity modeling and analysis, including, but not limited to, a visual decomposition of load intensity time-series into seasonal and trend parts, a simplified load intensity model as part of a model creation wizard, and an automated model instance extractor. As part of our LIMBO tutorial, we explain these features in detail. We demonstrate common use cases for LIMBO and show how they are supported. We also focus on LIMBOs extensible architecture and discuss how to use LIMBO as part of another tool-chain.

  @inproceedings{KiHeKo2014-SOSP-LIMBO, abstract = {{LIMBO is a tool for the creation of load profiles with variable intensity over time both from scratch and from existing data. Primarily, LIMBO's intended use is the description of load arrival behaviours in open workloads. Specifically, LIMBO can be employed for the creation of custom request or user arrival time-stamps or for the re-parameterization of existing traces. LIMBO bases on the Descartes Load Intensity Model (DLIM) for the formalized description of its load intensity profiles. The DLIM formalism can be understood as a structure for piece-wise defined and combined mathematical functions. We outline DLIM and its elements and demonstrate its integration within LIMBO. LIMBO is capable of generating request or user arrival time stamps from DLIM instances. In a next step, these generated time-stamps can be used for both open workload based benchmarking and simulations. The TimestampTimer plug-in for JMeter already allows the former. LIMBO also offers a range of tools for easy load intensity modeling and analysis, including, but not limited to, a visual decomposition of load intensity time-series into seasonal and trend parts, a simplified load intensity model as part of a model creation wizard, and an automated model instance extractor. As part of our LIMBO tutorial, we explain these features in detail. We demonstrate common use cases for LIMBO and show how they are supported. We also focus on LIMBOs extensible architecture and discuss how to use LIMBO as part of another tool-chain.}}, author = {von Kistowski, J{\'o}akim and Herbst, Nikolas Roman and Kounev, Samuel}, booktitle = {Proceedings of the Symposium on Software Performance 2014}, keywords = {t_poster}, month = 11, note = {<b>Best Poster Award</b>}, pages = {131–140}, publisher = {University of Stuttgart, Faculty of Computer Science, Electrical Engineering, and Information Technology}, title = {{Using and Extending LIMBO for the Descriptive Modeling of Arrival Behaviors}}, year = 2014 }
• 
  Experience Report: An Analysis of Hypercall Handler Vulnerabilities. A. Milenkoski; B. D. Payne; N. Antunes; M. Vieira; S. Kounev; in Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) --- Research Track (2014).

  Acceptance Rate: 25%, Best Paper Award Nomination
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.

  @inproceedings{MiPaAnViKo2014-ISSRE-AnAnalHypHanVulns, abstract = {{Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.}}, address = {{Washington DC, USA}}, author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel}, booktitle = {{Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) — Research Track}}, keywords = {Reliability}, month = 11, note = {Acceptance Rate: 25\%, <b>Best Paper Award Nomination</b>}, organization = {IEEE}, publisher = {IEEE Computer Society}, title = {{Experience Report: An Analysis of Hypercall Handler Vulnerabilities}}, year = 2014 }
• 
  Quantitative Evaluation of Service Dependability in Shared Execution Environments (Extended Abstract of Keynote Talk). S. Kounev; in Proceedings of the 11th International Conference on Quantitative Evaluation of SysTems (QEST 2014), Florence, Italy (2014).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{Ko2014-QEST-Keynote, author = {Kounev, Samuel}, booktitle = {Proceedings of the 11th International Conference on Quantitative Evaluation of SysTems (QEST 2014), Florence, Italy}, keywords = {Virtualization}, month = {{September}}, title = {{Quantitative Evaluation of Service Dependability in Shared Execution Environments (Extended Abstract of Keynote Talk)}}, year = 2014 }
• 
  Implementing Application-Aware Resource Allocation on a Home Gateway for the Example of YouTube. F. Wamser; L. Iffländer; T. Zinner; P. Tran-Gia; in International Conference on Mobile Networks and Management (2014). 301–312.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{wamser2014implementing, author = {Wamser, Florian and Iffländer, Lukas and Zinner, Thomas and Tran-Gia, Phuoc}, booktitle = {International Conference on Mobile Networks and Management}, keywords = {Application-aware}, month = 10, organization = {Springer International Publishing}, pages = {301–312}, title = {Implementing Application-Aware Resource Allocation on a Home Gateway for the Example of YouTube}, year = 2014 }
• 
  Demonstrating the Prospects of Dynamic Application-Aware Networking in a Home Environment. F. Wamser; T. Zinner; L. Iffländer; P. Tran-Gia; in Proceedings of the 2014 ACM conference on SIGCOMM (2014).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{WaZiIfTr-SIGCOMM-2014-demonstrating, author = {Wamser, Florian and Zinner, Thomas and Iffländer, Lukas and Tran-Gia, Phuoc}, booktitle = {Proceedings of the 2014 ACM conference on SIGCOMM}, keywords = {Optimization}, month = {08}, title = {Demonstrating the Prospects of Dynamic Application-Aware Networking in a Home Environment}, year = 2014 }
• 
  Remodularizing Legacy Model Transformations with Automatic Clustering Techniques. A. Rentschler; D. Werle; Q. Noorshams; L. Happe; R. Reussner; in Proceedings of the 3rd Workshop on the Analysis of Model Transformations (AMT@MODELS 2014), Valencia, Spain, September 29, 2014, B. Baudry, J. Dingel, L. Lucio, H. Vangheluwe (eds.) (2014).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{ReWeNoHaRe2014-AMT-Transformation, author = {Rentschler, Andreas and Werle, Dominik and Noorshams, Qais and Happe, Lucia and Reussner, Ralf}, booktitle = {Proceedings of the 3rd Workshop on the Analysis of Model Transformations (AMT@MODELS 2014), Valencia, Spain, September 29, 2014}, editor = {Baudry, Benoit and Dingel, J{\"u}rgen and Lucio, Levi and Vangheluwe, Hans}, keywords = {t_workshop}, month = 10, publisher = {CEUR-WS.org}, series = {CEUR Workshop Proceedings}, title = {{Remodularizing Legacy Model Transformations with Automatic Clustering Techniques}}, year = 2014 }
• 
  Model-Based Runtime Performance Management for Virtualized Applications. S. Spinner; S. Kounev; X. Zhu; L. Lu; M. Uysal; A. Holler; R. Griffith; P. Padala; in VMware R&D Innovation Offsite (RADIO) (2014).
  • [ BibTeX ]
  @inproceedings{SpinnerKZLUHGP-2014-RADIO-ModelBasedScaling, author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Holler, Anne and Griffith, Rean and Padala, Pradeep}, booktitle = {VMware R&D Innovation Offsite (RADIO)}, keywords = {se2}, month = {07}, title = {{Model-Based Runtime Performance Management for Virtualized Applications}}, year = 2014 }
• 
  Technical Information on Vulnerabilities of Hypercall Handlers A. Milenkoski; M. Vieira; B. D. Payne; N. Antunes; S. Kounev; (2014).
  • [ Abstract ]
  • [ BibTeX ]
  Modern virtualized service infrastructures expose attack vectors that enable attacks of high severity, such as attacks targeting hypervisors. A malicious user of a guest VM (virtual machine) may execute an attack against the underlying hypervisor via hypercalls, which are software traps from a kernel of a fully or partially paravirtualized guest VM to the hypervisor. The exploitation of a vulnerability of a hypercall handler may have severe consequences such as altering hypervisor's memory, which may result in the execution of malicious code with hypervisor privilege. Despite the importance of vulnerabilities of hypercall handlers, there is not much publicly available information on them. This significantly hinders advances towards securing hypercall interfaces. In this work, we provide in-depth technical information on publicly disclosed vulnerabilities of hypercall handlers. Our vulnerability analysis is based on reverse engineering the released patches fixing the considered vulnerabilities. For each analyzed vulnerability, we provide background information essential for understanding the vulnerability, and information on the vulnerable hypercall handler and the error causing the vulnerability. We also show how the vulnerability can be triggered and discuss the state of the targeted hypervisor after the vulnerability has been triggered.

  @techreport{MiViPaAnKo2014-TechReport-TechInformation, abstract = {{Modern virtualized service infrastructures expose attack vectors that enable attacks of high severity, such as attacks targeting hypervisors. A malicious user of a guest VM (virtual machine) may execute an attack against the underlying hypervisor via hypercalls, which are software traps from a kernel of a fully or partially paravirtualized guest VM to the hypervisor. The exploitation of a vulnerability of a hypercall handler may have severe consequences such as altering hypervisor's memory, which may result in the execution of malicious code with hypervisor privilege. Despite the importance of vulnerabilities of hypercall handlers, there is not much publicly available information on them. This significantly hinders advances towards securing hypercall interfaces. In this work, we provide in-depth technical information on publicly disclosed vulnerabilities of hypercall handlers. Our vulnerability analysis is based on reverse engineering the released patches fixing the considered vulnerabilities. For each analyzed vulnerability, we provide background information essential for understanding the vulnerability, and information on the vulnerable hypercall handler and the error causing the vulnerability. We also show how the vulnerability can be triggered and discuss the state of the targeted hypervisor after the vulnerability has been triggered.}}, address = {{7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}}, author = {Milenkoski, Aleksandar and Vieira, Marco and Payne, Bryan D. and Antunes, Nuno and Kounev, Samuel}, institution = {SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC)}, keywords = {Reliability}, month = {08}, title = {{Technical Information on Vulnerabilities of Hypercall Handlers}}, type = {{Technical Report SPEC-RG-2014-001 v.1.0}}, year = 2014 }
• 
  Architecture-Level Software Performance Models for Online Performance Prediction. F. Brosig; Thesis; Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany. (2014, July).

  Distinction
  • [ BibTeX ]
  • [ URL ]
  @phdthesis{Brosig2014-KIT-OnlinePerformancePrediction, author = {Brosig, Fabian}, keywords = {Prediction}, month = {07}, note = {<b>{Distinction}</b>}, school = {Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany}, title = {Architecture-Level Software Performance Models for Online Performance Prediction}, year = 2014 }
• 
  Architecture-Level Software Performance Abstractions for Online Performance Prediction. F. Brosig; N. Huber; S. Kounev; in Elsevier Science of Computer Programming Journal (SciCo) (2014). Vol. 90, Part B 71–92.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{BrHuKo2013-SciCo-SoftwarePerformanceAbstractions, author = {Brosig, Fabian and Huber, Nikolaus and Kounev, Samuel}, journal = {Elsevier Science of Computer Programming Journal (SciCo)}, keywords = {t_journalmagazine}, month = {09}, pages = {71–92}, publisher = {Elsevier}, title = {{Architecture-Level Software Performance Abstractions for Online Performance Prediction}}, volume = {Vol. 90, Part B}, year = 2014 }
• 
  Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation. S. Spinner; S. Kounev; X. Zhu; L. Lu; M. Uysal; A. Holler; R. Griffith; in Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO) (2014). 157–166.

  Acceptance Rate (Full Papers): 26%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.

  @inproceedings{SpKoZhLuUyHoGr2014-SASO, abstract = {Applications in virtualized data centers are often subject to Service Level Objectives (SLOs) regarding their performance (e.g., latency or throughput). In order to fulfill these SLOs, it is necessary to allocate sufficient resources of different types (CPU, memory, I/O, etc.) to an application. However, the relationship between the application performance and the resource allocations is complex and depends on multiple factors including application architecture, system configuration, and workload demands. In this paper, we present a model-based approach to ensure that the application performance meets the user-defined SLO efficiently by runtime "vertical scaling" (i.e., adding or removing resources) of individual virtual machines (VMs) running the application. A layered performance model describing the relationship between the resource allocations and the observed application performance is automatically extracted and updated online using resource demand estimation techniques. Such a model is then used in a feedback controller to dynamically adapt the number of virtual CPUs of individual VMs. We have implemented the controller on top of the VMware vSphere platform and evaluated it in a case study using a real-world email and groupware server. The experimental results show that our approach allows the managed application to achieve SLO satisfaction in spite of workload demand variation while avoiding oscillations commonly observed with state-of-the-art threshold-based controllers.}, author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Lu, Lei and Uysal, Mustafa and Holler, Anne and Griffith, Rean}, booktitle = {Proceedings of the 2014 IEEE Eighth International Conference on Self-Adaptive and Self-Organizing Systems (SASO)}, keywords = {Self-adaptive-systems}, month = {09}, note = {Acceptance Rate (Full Papers): 26%}, pages = {157–166}, publisher = {IEEE}, title = {{Runtime Vertical Scaling of Virtualized Applications via Online Model Estimation}}, year = 2014 }
• 
  Quantitative Evaluation of Service Dependability in Shared Execution Environments. S. Kounev; (2014, September).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Florence15, author = {Kounev, Samuel}, howpublished = {Keynote Talk at FLORENCE 2014 - a joint scientific event of QEST 2014, SAFECOMP 2014, FORMATS 2014, EPEW 2014 and FMICS 2014, Florence, Italy}, keywords = {Talks}, month = {09}, title = {{Quantitative Evaluation of Service Dependability in Shared Execution Environments}}, year = 2014 }
• 
  Descartes Network Infrastructures (DNI) Manual: Meta-models, Transformations, Examples P. Rygielski; S. Kounev; (2014).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The goal of this document is to present the DNI Meta-Model, its instances, sub-meta-models, and model transformations in more details than in an usual conference or journal paper. We describe the meta-models and the transformations in details, give example of systems modeled using the meta-models, and present the transformations in a step-by-step manner.

  @techreport{RyKo-DNI-Manual-TR, abstract = {The goal of this document is to present the DNI Meta-Model, its instances, sub-meta-models, and model transformations in more details than in an usual conference or journal paper. We describe the meta-models and the transformations in details, give example of systems modeled using the meta-models, and present the transformations in a step-by-step manner.}, address = {Am Hubland, 97074 W{\"u}rzburg}, author = {Rygielski, Piotr and Kounev, Samuel}, institution = {Chair of Software Engineering, University of W{\"u}rzburg}, keywords = {DNI}, month = {09}, title = {{Descartes Network Infrastructures (DNI) Manual: Meta-models, Transformations, Examples}}, type = {{Technical Report v.0.3}}, year = 2014 }

• 
  Variants of the general interval power function. O. Heimlich; M. Nehmeier; J. Wolff von Gudenberg; in Soft Computing (2013). 17(8) 1357–1366.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The article gives an overview of power function variants and compares three differently extensive versions for use in interval arithmetics. Aiming at the general power function for inclusion in interval libraries, which is defined for as many pairs of base and exponent as possible, we observe that the definition of a power for each such pair depends on the context. This problem eventually comes up at the point where reasonable doubts about the definition 0_sup_0 and powers with negative base in correlation with a non-integral exponent arise. We come up with several variants serving distinct purposes, yet also recommend a general-purpose power function, which is unique for being restricted to integral exponents on the domain of negative bases. Three different variants of interval power functions satisfy all meaningful general exponentiation needs. We provide a unified treatment to handle the variants and present valuable implementation techniques for the computation of these interval functions along with a mathematic foundation.

  @article{intervalPowerVariants, abstract = {The article gives an overview of power function variants and compares three differently extensive versions for use in interval arithmetics. Aiming at the general power function for inclusion in interval libraries, which is defined for as many pairs of base and exponent as possible, we observe that the definition of a power for each such pair depends on the context. This problem eventually comes up at the point where reasonable doubts about the definition 0_sup_0 and powers with negative base in correlation with a non-integral exponent arise. We come up with several variants serving distinct purposes, yet also recommend a general-purpose power function, which is unique for being restricted to integral exponents on the domain of negative bases. Three different variants of interval power functions satisfy all meaningful general exponentiation needs. We provide a unified treatment to handle the variants and present valuable implementation techniques for the computation of these interval functions along with a mathematic foundation.}, author = {Heimlich, Oliver and Nehmeier, Marco and Wolff von Gudenberg, J{\"u}rgen}, journal = {Soft Computing}, keywords = {Reliability}, number = 8, pages = {1357-1366}, publisher = {Springer Berlin Heidelberg}, title = {Variants of the general interval power function}, volume = 17, year = 2013 }
• 
  Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions. C. Rathfelder; B. Klatt; K. Sachs; S. Kounev; in Journal of Software and Systems Modeling (SoSyM) (2013). 1–27.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @article{rathfelder2013a, author = {Rathfelder, Christoph and Klatt, Benjamin and Sachs, Kai and Kounev, Samuel}, journal = {Journal of Software and Systems Modeling (SoSyM)}, keywords = {t_journalmagazine}, month = {03}, pages = {1–27}, publisher = {Springer Verlag}, title = {Modeling Event-based Communication in Component-based Software Architectures for Performance Predictions}, year = 2013 }
• 
  Ingenieurmässige Entwicklung verlässlicher und effizienter Software-Systeme. S. Kounev; (2013, March).
  • [ BibTeX ]
  @misc{talk-Kounev-WueInv13, author = {Kounev, Samuel}, howpublished = {Invited Talk, Faculty of Mathematics and Computer Science, University of W{\"u}rzburg, W{\"u}rzburg}, keywords = {Talks}, month = {03}, title = {{Ingenieurm\"{a}ssige Entwicklung verl\"{a}sslicher und effizienter Software-Systeme}}, year = 2013 }
• 
  The Descartes Meta-Model (DMM) - An Architecture-level Modeling Language for Online Quality-of-Service and Resource Management. S. Kounev; (2013, March).
  • [ BibTeX ]
  @misc{talk-Kounev-DMMDublin13, author = {Kounev, Samuel}, howpublished = {Invited Talk. UCD School of Computer Science and Informatics, University College Dublin, Dublin, Ireland}, keywords = {Talks}, month = {03}, title = {{The Descartes Meta-Model (DMM) - An Architecture-level Modeling Language for Online Quality-of-Service and Resource Management}}, year = 2013 }
• 
  Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey. P. Rygielski; S. Kounev; in PIK --- Praxis der Informationsverarbeitung und Kommunikation (2013). 36(1) 55–64.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.

  @article{RyKo2013, abstract = {The increasing popularity of Cloud Computing is leading to the emergence of large virtualized data centers hosting increasingly complex and dynamic IT systems and services. Over the past decade, the efficient sharing of computational resources through virtualization has been subject to intensive research, while network management in cloud data centers has received less attention. A variety of network-intensive applications require QoS (Quality-of-Service) provisioning, performance isolation and support for flexible and efficient migration of virtual machines. In this paper, we survey existing network virtualization approaches and evaluate the extent to which they can be used as a basis for realizing the mentioned requirements in a cloud data center. More specifically, we identify generic network virtualization techniques, characterize them according to their features related to QoS management and performance isolation, and show how they can be composed together and used as building blocks for complex network virtualization solutions. We then present an overview of selected representative cloud platforms and show how they leverage the generic techniques as a basis for network resource management. Finally, we outline open issues and research challenges in the area of performance modeling and proactive resource management of virtualized data center infrastructures.}, author = {Rygielski, Piotr and Kounev, Samuel}, journal = {PIK — Praxis der Informationsverarbeitung und Kommunikation}, keywords = {SDN}, month = {02}, number = 1, pages = {55–64}, publisher = {de Gruyter}, title = {{Network Virtualization for QoS-Aware Resource Management in Cloud Data Centers: A Survey}}, volume = 36, year = 2013 }
• 
  Probabilistic tenant selection in quota enforcing Request Admission mechanisms for Shared Applications. A. Mehta; R. Krebs; (2013).
  • [ BibTeX ]
  @misc{Kr2014-uspatent-Probabilistic, author = {Mehta, Arpit and Krebs, Rouven}, howpublished = {US Patent Application No 13/904,729}, keywords = {se2}, title = {Probabilistic tenant selection in quota enforcing Request Admission mechanisms for Shared Applications}, year = 2013 }
• 
  Predictive Performance Modeling of Virtualized Storage Systems using Optimized Statistical Regression Techniques. Q. Noorshams; D. Bruhn; S. Kounev; R. Reussner; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (ICPE 2013) (2013). 283–294.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoBrKoRe2013-ICPE-IORegressionModels, address = {New York, NY, USA}, author = {Noorshams, Qais and Bruhn, Dominik and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering (ICPE 2013)}, keywords = {Optimization}, pages = {283–294}, publisher = {ACM}, series = {ICPE'13}, title = {{Predictive Performance Modeling of Virtualized Storage Systems using Optimized Statistical Regression Techniques}}, year = 2013 }
• 
  Modelling Event-Based Interactions in Component-Based Architectures for Quantitative System Evaluation. C. Rathfelder; Thesis; KIT Scientific Publishing. (2013).
  • [ BibTeX ]
  @phdthesis{Rathfelder2013_1000032232, author = {Rathfelder, Christoph}, keywords = {descartes}, publisher = {{KIT Scientific Publishing}}, series = {The Karlsruhe Series on Software Design and Quality / Ed. by Prof. Dr. Ralf Reussner}, title = {Modelling Event-Based Interactions in Component-Based Architectures for Quantitative System Evaluation}, volume = 10, year = 2013 }
• 
  Parallel Interval Newton Method on CUDA. P.-D. Beck; M. Nehmeier; in Applied Parallel and Scientific Computing, P. Manninen, P. Öster (eds.) (2013). (Vol. 7782) 454–464.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this paper we discuss a parallel variant of the interval Newton method for root finding of non linear continuously differentiable functions on the CUDA architecture. For this purpose we have investigated different dynamic load balancing methods to get an evenly balanced workload during the parallel computation. We tested the functionality, correctness and performance of our implementation in different case studies and compared it with other implementations.

  @incollection{nehmeierPara2012, abstract = {In this paper we discuss a parallel variant of the interval Newton method for root finding of non linear continuously differentiable functions on the CUDA architecture. For this purpose we have investigated different dynamic load balancing methods to get an evenly balanced workload during the parallel computation. We tested the functionality, correctness and performance of our implementation in different case studies and compared it with other implementations.}, author = {Beck, Philip-Daniel and Nehmeier, Marco}, booktitle = {Applied Parallel and Scientific Computing}, editor = {Manninen, Pekka and {\"O}ster, Per}, keywords = {Reliability}, pages = {454-464}, publisher = {Springer Berlin Heidelberg}, series = {Lecture Notes in Computer Science}, title = {{P}arallel {I}nterval {N}ewton {M}ethod on {CUDA}}, volume = 7782, year = 2013 }
• 
  Deriving performance-relevant infrastructure properties through model-based experiments with Ginpex. M. Hauck; M. Kuperberg; N. Huber; R. Reussner; in Software & Systems Modeling (2013). 1–21.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  To predict the performance of an application, it is crucial to consider the performance of the underlying infrastructure. Thus, to yield accurate prediction results, performance-relevant properties and behaviour of the infrastructure have to be integrated into performance models. However, capturing these properties is a cumbersome and error-prone task, as it requires carefully engineered measurements and experiments. Existing approaches for creating infrastructure performance models require manual coding of these experiments, or ignore the detailed properties in the models. The contribution of this paper is the Ginpex approach, which introduces goal-oriented and model-based specification and generation of executable performance experiments for automatically detecting and quantifying performance-relevant infrastructure properties. Ginpex provides a metamodel for experiment specification and comes with predefined experiment templates that provide automated experiment execution on the target platform and also automate the evaluation of the experiment results. We evaluate Ginpex using three case studies, where experiments are executed to quantify various infrastructure properties.

  @article{hauck2013a, abstract = {To predict the performance of an application, it is crucial to consider the performance of the underlying infrastructure. Thus, to yield accurate prediction results, performance-relevant properties and behaviour of the infrastructure have to be integrated into performance models. However, capturing these properties is a cumbersome and error-prone task, as it requires carefully engineered measurements and experiments. Existing approaches for creating infrastructure performance models require manual coding of these experiments, or ignore the detailed properties in the models. The contribution of this paper is the Ginpex approach, which introduces goal-oriented and model-based specification and generation of executable performance experiments for automatically detecting and quantifying performance-relevant infrastructure properties. Ginpex provides a metamodel for experiment specification and comes with predefined experiment templates that provide automated experiment execution on the target platform and also automate the evaluation of the experiment results. We evaluate Ginpex using three case studies, where experiments are executed to quantify various infrastructure properties.}, author = {Hauck, Michael and Kuperberg, Michael and Huber, Nikolaus and Reussner, Ralf}, journal = {Software & Systems Modeling}, keywords = {Virtualization}, pages = {1-21}, publisher = {Springer-Verlag}, title = {Deriving performance-relevant infrastructure properties through model-based experiments with Ginpex}, year = 2013 }
• 
  Soft Reservations: Uncertainty-Aware Resource Reservations in IaaS Environments. S. V. Mohammadi; S. Kounev; A. Juan-Verdejo; B. Surajbali; in Proceedings of the 3rd International Symposium on Business Modeling and Software Design (BMSD 2013) (2013).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{MoKoJuBa2013-BMSD-SoftReservations, author = {Mohammadi, Seyed Vahid and Kounev, Samuel and Juan-Verdejo, Adri{\"a}n and Surajbali, Bholanathsingh}, booktitle = {Proceedings of the 3rd International Symposium on Business Modeling and Software Design (BMSD 2013)}, keywords = {Optimization}, title = {{Soft Reservations: Uncertainty-Aware Resource Reservations in IaaS Environments}}, year = 2013 }
• 
  Performance modeling and analysis of message-oriented event-driven systems. K. Sachs; S. Kounev; A. Buchmann; in Journal of Software and Systems Modeling (2013). 12(4) 705–729.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.

  @article{SaKoBu2011-SoSyM-PerfModMoEdSys, abstract = {Message-oriented event-driven systems are becoming increasingly ubiquitous in many industry domains including telecommunications, transportation and supply chain management. Applications in these areas typically have stringent requirements for performance and scalability. To guarantee adequate quality-of-service, systems must be subjected to a rigorous performance and scalability analysis before they are put into production. In this paper, we present a comprehensive modeling methodology for message-oriented event-driven systems in the context of a case study of a representative application in the supply chain management domain. The methodology, which is based on queueing Petri nets, provides a basis for performance analysis and capacity planning. We study a deployment of the SPECjms2007 standard benchmark on a leading commercial middleware platform. A detailed system model is built in a step-by-step fashion and then used to predict the system performance under various workload and configuration scenarios. After the case study, we present a set of generic performance modeling patterns that can be used as building blocks when modeling message-oriented event-driven systems. The results demonstrate the effectiveness, practicality and accuracy of the proposed modeling and prediction approach.}, author = {Sachs, Kai and Kounev, Samuel and Buchmann, Alejandro}, journal = {Journal of Software and Systems Modeling}, keywords = {QPN}, number = 4, pages = {705–729}, publisher = {Springer-Verlag}, title = {Performance modeling and analysis of message-oriented event-driven systems}, volume = 12, year = 2013 }
• 
  Modelling Event-Based Interactions in Component-Based Architectures for Quantitative System Evaluation C. Rathfelder; in The Karlsruhe Series on Software Design and Quality (2013). (Vol. 10) KIT Scientific Publishing, Karlsruhe, Germany.
  • [ BibTeX ]
  • [ URL ]
  @proceedings{rathfelder2013b, address = {Karlsruhe, Germany}, author = {Rathfelder, Christoph}, keywords = {t_book}, publisher = {KIT Scientific Publishing}, series = {The Karlsruhe Series on Software Design and Quality}, title = {Modelling Event-Based Interactions in Component-Based Architectures for Quantitative System Evaluation}, volume = 10, year = 2013 }
• 
  Multi-Stage Feedback controller for prioritizing tenants for Multi-Tenant application. R. Krebs; A. Mehta; (2013).
  • [ BibTeX ]
  @misc{Kr2014-uspatent-Controller, author = {Krebs, Rouven and Mehta, Arpit}, howpublished = {US Patent Application No 13/972,477}, keywords = {se2}, title = {Multi-Stage Feedback controller for prioritizing tenants for Multi-Tenant application}, year = 2013 }
• 
  ERC Starting Grant Finalist. S. Kounev; (2013).
  • [ BibTeX ]
  @misc{talk-Kounev-ERCFinal, author = {Kounev, Samuel}, howpublished = {(top 18\% out of 3.329 StG applicants), European Research Council, Brussels, Belgium}, keywords = {Talks}, title = {{ERC Starting Grant Finalist}}, year = 2013 }
• 
  A Generic Approach for Architecture-level Performance Modeling and Prediction of Virtualized Storage Systems. Q. Noorshams; A. Rentschler; S. Kounev; R. Reussner; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2013). 339–342.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoReKoRe2013-ICPE-WIP, address = {New York, NY, USA}, author = {Noorshams, Qais and Rentschler, Andreas and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {t_wip}, pages = {339–342}, publisher = {ACM}, series = {ICPE '13}, title = {{A Generic Approach for Architecture-level Performance Modeling and Prediction of Virtualized Storage Systems}}, year = 2013 }
• 
  I/O Performance Modeling of Virtualized Storage Systems. Q. Noorshams; K. Rostami; S. Kounev; P. Tuma; R. Reussner; in Proceedings of the 21st IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013) (2013). 121–130.

  Acceptance Rate (Full Paper): 44/163 = 27%
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoRoKoTuRe2013-MASCOTS-IOQNs, author = {Noorshams, Qais and Rostami, Kiana and Kounev, Samuel and T{u}ma, Petr and Reussner, Ralf}, booktitle = {Proceedings of the 21st IEEE International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013)}, keywords = {Virtualization}, note = {Acceptance Rate (Full Paper): 44/163 = 27\%}, pages = {121-130}, series = {MASCOTS'13}, title = {{I/O Performance Modeling of Virtualized Storage Systems}}, year = 2013 }
• 
  A Feedback Controlled Scheduler for Performance Isolation in Multi-tenant Applications. R. Krebs; A. Mehta; in Proceedings of the 3rd IEEE International Conference on Cloud and Green Computing (CGC 2013) (2013).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KrLoKo2013-CGC-FeedbackController, author = {Krebs, Rouven and Mehta, Arpit}, booktitle = {Proceedings of the 3rd IEEE International Conference on Cloud and Green Computing (CGC 2013)}, keywords = {t_poster}, title = {{A Feedback Controlled Scheduler for Performance Isolation in Multi-tenant Applications}}, year = 2013 }
• 
  Application Level Based Resource Management in Multi-Tenant Applications. R. Krebs; A. Mehta; (2013).
  • [ BibTeX ]
  @misc{Kr2014-uspatent-ResourceControl, author = {Krebs, Rouven and Mehta, Arpit}, howpublished = {US Patent Application No 14/079,289}, keywords = {se2}, title = {Application Level Based Resource Management in Multi-Tenant Applications}, year = 2013 }
• 
  HInjector: Injecting Hypercall Attacks for Evaluating VMI-based Intrusion Detection Systems. A. Milenkoski; B. D. Payne; N. Antunes; M. Vieira; S. Kounev; in The 2013 Annual Computer Security Applications Conference (ACSAC 2013) (2013).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{MiPaAnViKo2013-ACSAC-HInjector, address = {Maryland, USA}, author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel}, booktitle = {The 2013 Annual Computer Security Applications Conference (ACSAC 2013)}, keywords = {Reliability}, month = {03}, publisher = {{Applied Computer Security Associates (ACSA)}}, title = {{HInjector: Injecting Hypercall Attacks for Evaluating VMI-based Intrusion Detection Systems}}, year = 2013 }
• 
  Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013) K. Sachs; (S. Kounev; S. Zschaler; K. Sachs, eds.) (2013). ACM.
  • [ BibTeX ]
  @proceedings{KoZsSa2013-HOTTOPICS-Proceedings, author = {Sachs, Kai}, editor = {Kounev, Samuel and Zschaler, Steffen and Sachs, Kai}, keywords = {t_workshop}, month = {04}, publisher = {ACM}, title = {Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)}, year = 2013 }
• 
  RELATE: A Research Training Network on Engineering and Provisioning of Service-Based Cloud Applications. S. Kounev; (2013, April).
  • [ BibTeX ]
  @misc{talk-Kounev-hottopics13, author = {Kounev, Samuel}, howpublished = {Invited Talk, International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013), Prague, Czech Republic}, keywords = {Talks}, month = {04}, title = {{RELATE: A Research Training Network on Engineering and Provisioning of Service-Based Cloud Applications}}, year = 2013 }
• 
  Online model-based performance and resource management in virtualized application environments. S. Kounev; (2013, July).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-VMWare13, author = {Kounev, Samuel}, howpublished = {Invited Talk. 2013 VMware Academic Research Symposium, Palo Alto, CA, USA}, keywords = {Talks}, month = {07}, title = {{Online model-based performance and resource management in virtualized application environments}}, year = 2013 }
• 
  RELATE: A Research Training Network on Engineering and Provisioning of Service-Based Cloud Applications. S. Kounev; S. Rizou; S. Zschaler; S. Alexakis; T. Bures; J.-M. Jézéquel; D. Kourtesis; S. Pantelopoulos; in International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013) (2013).
  • [ BibTeX ]
  @inproceedings{Ko2013-HotTopiCS-Relate, author = {Kounev, Samuel and Rizou, Stamatia and Zschaler, Steffen and Alexakis, Spiros and Bures, Tomas and J{\'e}z{\'e}quel, Jean-Marc and Kourtesis, Dimitrios and Pantelopoulos, Stelios}, booktitle = {International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)}, keywords = {t_workshop}, month = {04}, title = {{RELATE: A Research Training Network on Engineering and Provisioning of Service-Based Cloud Applications}}, year = 2013 }
• 
  Model-Based Throughput Prediction in Data Center Networks. P. Rygielski; S. Kounev; S. Zschaler; in Proceedings of the 2nd IEEE International Workshop on Measurements and Networking (M\&N 2013) (2013). 167–172.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper, we address the problem of performance analysis in computer networks. We present a new meta-model designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. We evaluate the approach in a case study of a road traffic monitoring system. We compare the performance prediction results against the real system and a benchmark. The presented results show that our approach, despite of introducing many modeling abstractions, delivers predictions with errors less than 32% and correctly detects bottlenecks in the modeled network.

  @inproceedings{RyKoZs2013-ThroughputPrediction, abstract = {In this paper, we address the problem of performance analysis in computer networks. We present a new meta-model designed for the performance modeling of network infrastructures in modern data centers. Instances of our metamodel can be automatically transformed into stochastic simulation models for performance prediction. We evaluate the approach in a case study of a road traffic monitoring system. We compare the performance prediction results against the real system and a benchmark. The presented results show that our approach, despite of introducing many modeling abstractions, delivers predictions with errors less than 32% and correctly detects bottlenecks in the modeled network.}, author = {Rygielski, Piotr and Kounev, Samuel and Zschaler, Steffen}, booktitle = {{Proceedings of the 2nd IEEE International Workshop on Measurements and Networking (M\&N 2013)}}, keywords = {DNI}, month = 10, pages = {167–172}, title = {{Model-Based Throughput Prediction in Data Center Networks}}, year = 2013 }
• 
  Towards Online Performance Model Extraction in Virtualized Environments. S. Spinner; S. Kounev; X. Zhu; M. Uysal; in Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013), N. Bencomo, R. France, S. Götz, B. Rumpe (eds.) (2013). 89–95.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.

  @inproceedings{SpKoZhUy2013-mrt-TowardsModelExtraction, abstract = {Virtualization increases the complexity and dynamics of modern software architectures making it a major challenge to manage the end-to-end performance of applications. Architecture-level performance models can help here as they provide the modeling power and analysis fexibility to predict the performance behavior of applications under varying workloads and configurations. However, the construction of such models is a complex and time-consuming task. In this position paper, we discuss how the existing concept of virtual appliances can be extended to automate the extraction of architecture-level performance models during system operation.}, author = {Spinner, Simon and Kounev, Samuel and Zhu, Xiaoyun and Uysal, Mustafa}, booktitle = {Proceedings of the 8th Workshop on Models @ Run.time (MRT 2013) co-located with ACM/IEEE 16th International Conference on Model Driven Engineering Languages and Systems (MODELS 2013)}, editor = {Bencomo, Nelly and France, Robert and G{\"o}tz, Sebastian and Rumpe, Bernhard}, keywords = {Virtualization}, month = {09}, pages = {89–95}, publisher = {CEUR-WS}, title = {{Towards Online Performance Model Extraction in Virtualized Environments}}, year = 2013 }
• 
  Using Queuing Models for Large System Migration Scenarios -- An Industrial Case Study with IBM System z. R. Vaupel; Q. Noorshams; S. Kounev; R. Reussner; in Computer Performance Engineering, Proceedings of the 10th European Performance Engineering Workshop (EPEW 2013), Venice, Italy, M. S. Balsamo, W. J. Knottenbelt, A. Marin (eds.) (2013). (Vol. 8168) 263–275.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @incollection{VaNoKoRe2013-EPEW-IndustrialCaseStudySystemz, author = {Vaupel, Robert and Noorshams, Qais and Kounev, Samuel and Reussner, Ralf}, booktitle = {Computer Performance Engineering, Proceedings of the 10th European Performance Engineering Workshop (EPEW 2013), Venice, Italy}, editor = {Balsamo, Maria Simonetta and Knottenbelt, William J. and Marin, Andrea}, keywords = {t_workshop}, month = {09}, pages = {263-275}, publisher = {Springer Berlin Heidelberg}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Using Queuing Models for Large System Migration Scenarios – An Industrial Case Study with IBM System z}}, volume = 8168, year = 2013 }
• 
  Evaluating Approaches for Performance Prediction in Virtualized Environments. F. Brosig; F. Gorsler; N. Huber; S. Kounev; in Proceedings of the IEEE 21st International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013) (2013).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{BrGoHuKo2013-MASCOTS-EvaluationApproachesForPerformancePredictionInVirtualizedEnvironments, author = {Brosig, Fabian and Gorsler, Fabian and Huber, Nikolaus and Kounev, Samuel}, booktitle = {Proceedings of the IEEE 21st International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2013)}, keywords = {Virtualization}, month = {08}, title = {{Evaluating Approaches for Performance Prediction in Virtualized Environments}}, year = 2013 }
• 
  Online workload forecasting and performance prediction for elastic capacity management in cloud environments. S. Kounev; (2013, July).
  • [ BibTeX ]
  @misc{talk-Kounev-Salesforce13, author = {Kounev, Samuel}, howpublished = {Invited Talk. Salesforce.com, San Francisco, CA, USA}, keywords = {Talks}, month = {07}, title = {{Online workload forecasting and performance prediction for elastic capacity management in cloud environments}}, year = 2013 }
• 
  Experimental Evaluation of the Performance-Influencing Factors of Virtualized Storage Systems. Q. Noorshams; S. Kounev; R. Reussner; in Computer Performance Engineering. 9th European Performance Engineering Workshop (EPEW 2012), Munich, Germany, July 30, 2012, and 28th UK Performance Engineering Workshop (UKPEW 2012), Edinburgh, UK, July 2, 2012, Revised Selected Papers, M. Tribastone, S. Gilmore (eds.) (2013). (Vol. 7587) 63–79.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @incollection{NoKoRe2012-EPEW-Experiments, author = {Noorshams, Qais and Kounev, Samuel and Reussner, Ralf}, booktitle = {Computer Performance Engineering. 9th European Performance Engineering Workshop (EPEW 2012), Munich, Germany, July 30, 2012, and 28th UK Performance Engineering Workshop (UKPEW 2012), Edinburgh, UK, July 2, 2012, Revised Selected Papers}, editor = {Tribastone, Mirco and Gilmore, Stephen}, keywords = {Virtualization}, month = {07}, pages = {63-79}, publisher = {Springer Berlin Heidelberg}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Experimental Evaluation of the Performance-Influencing Factors of Virtualized Storage Systems}}, volume = 7587, year = 2013 }
• 
  Multi-Tenancy Performance Benchmark for Web Application Platforms. R. Krebs; A. Wert; S. Kounev; in Proceedings of the 13th International Conference on Web Engineering (ICWE 2013) (2013).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KrWeKo2013-icwe-MTBenchmark, author = {Krebs, Rouven and Wert, Alexander and Kounev, Samuel}, booktitle = {{Proceedings of the 13th International Conference on Web Engineering (ICWE 2013)}}, keywords = {Virtualization}, month = {07}, organization = {Aalborg University, Denmark}, publisher = {Springer-Verlag}, title = {{Multi-Tenancy Performance Benchmark for Web Application Platforms}}, year = 2013 }
• 
  Controlling the Palladio Bench using the Descartes Query Language. F. Gorsler; F. Brosig; S. Kounev; in Proceedings of the Symposium on Software Performance: Joint Kieker/Palladio Days (KPDAYS 2013), S. Becker, W. Hasselbring, A. van Hoorn, R. Reussner (eds.) (2013). 109–118.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The Palladio Bench is a tool to model, simulate and analyze Palladio Component Model (PCM) instances. However, for the Palladio Bench, no single interface to automate experiments or Application Programming Interface (API) to trigger the simulation of PCM instances and to extract performance prediction results is available. The Descartes Query Language (DQL) is a novel approach of a declarative query language to integrate different performance modeling and prediction techniques behind a unifying interface. Users benefit from the abstraction of specific tools to prepare and trigger performance predictions, less effort to obtain performance metrics of interest, and means to automate performance predictions. In this paper, we describe the realization of a DQL Connector for PCM and demonstrate the applicability of our approach in a case study.

  @inproceedings{GoBrKo2013-ControllingPCM, abstract = {The Palladio Bench is a tool to model, simulate and analyze Palladio Component Model (PCM) instances. However, for the Palladio Bench, no single interface to automate experiments or Application Programming Interface (API) to trigger the simulation of PCM instances and to extract performance prediction results is available. The Descartes Query Language (DQL) is a novel approach of a declarative query language to integrate different performance modeling and prediction techniques behind a unifying interface. Users benefit from the abstraction of specific tools to prepare and trigger performance predictions, less effort to obtain performance metrics of interest, and means to automate performance predictions. In this paper, we describe the realization of a DQL Connector for PCM and demonstrate the applicability of our approach in a case study.}, address = {Aachen, Germany}, author = {Gorsler, Fabian and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the Symposium on Software Performance: Joint Kieker/Palladio Days (KPDAYS 2013)}, editor = {Becker, Steffen and Hasselbring, Wilhelm and van Hoorn, Andr{\'e} and Reussner, Ralf}, keywords = {Optimization}, month = 11, number = 1083, pages = {109–118}, publisher = {CEUR-WS.org}, title = {Controlling the Palladio Bench using the Descartes Query Language}, year = 2013 }
• 
  Interactive Visual Analytics for Efficient Maintenance of Model Transformations. A. Rentschler; Q. Noorshams; L. Happe; R. Reussner; in Proceedings of the 6th International Conference on Model Transformation (ICMT ’13), Budapest, Hungary, K. Duddy, G. Kappel (eds.) (2013). (Vol. 7909) 141–157.

  Acceptance Rate: 20.7%
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{ReNoHaRe2013-ICMT-VisualAnalystics, author = {Rentschler, Andreas and Noorshams, Qais and Happe, Lucia and Reussner, Ralf}, booktitle = {Proceedings of the 6th International Conference on Model Transformation (ICMT '13), Budapest, Hungary}, editor = {Duddy, Keith and Kappel, Gerti}, keywords = {se2}, month = {06}, note = {Acceptance Rate: 20.7\%}, pages = {141–157}, publisher = {Springer-Verlag Berlin Heidelberg}, series = {Lecture Notes in Computer Science}, title = {{Interactive Visual Analytics for Efficient Maintenance of Model Transformations}}, volume = 7909, year = 2013 }
• 
  Cloud Usage Patterns: A Formalism for Description of Cloud Usage Scenarios A. Milenkoski; A. Iosup; S. Kounev; K. Sachs; P. Rygielski; J. Ding; W. Cirne; F. Rosenberg; (2013).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Cloud computing is becoming an increasingly lucrative branch of the existing information and communication technologies (ICT). Enabling a debate about cloud usage scenarios can help with attracting new customers, sharing best-practices, and designing new cloud services. In contrast to previous approaches, which have attempted mainly to formalize the common service delivery models (i.e., Infrastructure-as-a-Service, Platform-as-a-Service, and Software-as-a-Service), in this work, we propose a formalism for describing common cloud usage scenarios referred to as cloud usage patterns. Our formalism takes a structuralist approach allowing decomposition of a cloud usage scenario into elements corresponding to the common cloud service delivery models. Furthermore, our formalism considers several cloud usage patterns that have recently emerged, such as hybrid services and value chains in which mediators are involved, also referred to as value chains with mediators. We propose a simple yet expressive textual and visual language for our formalism, and we show how it can be used in practice for describing a variety of real-world cloud usage scenarios. The scenarios for which we demonstrate our formalism include resource provisioning of global providers of infrastructure and/or platform resources, online social networking services, user-data processing services, online customer and ticketing services, online asset management and banking applications, CRM (Customer Relationship Management) applications, and online social gaming applications.

  @techreport{MiIoKoSaRyDiCiRo2013-TechReport-CloudUsagePatterns, abstract = {{Cloud computing is becoming an increasingly lucrative branch of the existing information and communication technologies (ICT). Enabling a debate about cloud usage scenarios can help with attracting new customers, sharing best-practices, and designing new cloud services. In contrast to previous approaches, which have attempted mainly to formalize the common service delivery models (i.e., Infrastructure-as-a-Service, Platform-as-a-Service, and Software-as-a-Service), in this work, we propose a formalism for describing common cloud usage scenarios referred to as cloud usage patterns. Our formalism takes a structuralist approach allowing decomposition of a cloud usage scenario into elements corresponding to the common cloud service delivery models. Furthermore, our formalism considers several cloud usage patterns that have recently emerged, such as hybrid services and value chains in which mediators are involved, also referred to as value chains with mediators. We propose a simple yet expressive textual and visual language for our formalism, and we show how it can be used in practice for describing a variety of real-world cloud usage scenarios. The scenarios for which we demonstrate our formalism include resource provisioning of global providers of infrastructure and/or platform resources, online social networking services, user-data processing services, online customer and ticketing services, online asset management and banking applications, CRM (Customer Relationship Management) applications, and online social gaming applications.}}, address = {{7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}}, author = {Milenkoski, Aleksandar and Iosup, Alexandru and Kounev, Samuel and Sachs, Kai and Rygielski, Piotr and Ding, Jason and Cirne, Walfredo and Rosenberg, Florian}, institution = {SPEC Research Group - Cloud Working Group, Standard Performance Evaluation Corporation (SPEC)}, keywords = {Virtualization}, month = {05}, title = {{Cloud Usage Patterns: A Formalism for Description of Cloud Usage Scenarios}}, type = {{Technical Report SPEC-RG-2013-001 v.1.0.1}}, year = 2013 }
• 
  Elasticity in Cloud Computing: What it is, and What it is Not. N. R. Herbst; S. Kounev; R. Reussner; in Proceedings of the 10th International Conference on Autonomic Computing (ICAC 2013) (2013).

  Acceptance Rate (Short Paper): 36.9%
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Originating from the field of physics and economics, the term elasticity is nowadays heavily used in the context of cloud computing. In this context, elasticity is commonly understood as the ability of a system to automatically provision and de-provision computing resources on demand as workloads change. However, elasticity still lacks a precise definition as well as representative metrics coupled with a benchmarking methodology to enable comparability of systems. Existing definitions of elasticity are largely inconsistent and unspecific leading to confusion in the use of the term and its differentiation from related terms such as scalability and efficiency; the proposed measurement methodologies do not provide means to quantify elasticity without mixing it with efficiency or scalability aspects. In this short paper, we propose a precise definition of elasticity and analyze its core properties and requirements explicitly distinguishing from related terms such as scalability, efficiency, and agility. Furthermore, we present a set of appropriate elasticity metrics and sketch a new elasticity tailored benchmarking methodology addressing the special requirements on workload design and calibration.

  @inproceedings{HeKoRe2013-ICAC-Elasticity, abstract = {{Originating from the field of physics and economics, the term elasticity is nowadays heavily used in the context of cloud computing. In this context, elasticity is commonly understood as the ability of a system to automatically provision and de-provision computing resources on demand as workloads change. However, elasticity still lacks a precise definition as well as representative metrics coupled with a benchmarking methodology to enable comparability of systems. Existing definitions of elasticity are largely inconsistent and unspecific leading to confusion in the use of the term and its differentiation from related terms such as scalability and efficiency; the proposed measurement methodologies do not provide means to quantify elasticity without mixing it with efficiency or scalability aspects. In this short paper, we propose a precise definition of elasticity and analyze its core properties and requirements explicitly distinguishing from related terms such as scalability, efficiency, and agility. Furthermore, we present a set of appropriate elasticity metrics and sketch a new elasticity tailored benchmarking methodology addressing the special requirements on workload design and calibration.}}, author = {Herbst, Nikolas Roman and Kounev, Samuel and Reussner, Ralf}, booktitle = {Proceedings of the 10th International Conference on Autonomic Computing (ICAC 2013)}, keywords = {Self-adaptive-systems}, month = {06}, note = {Acceptance Rate (Short Paper): 36.9\%}, publisher = {USENIX}, title = {{Elasticity in Cloud Computing: What it is, and What it is Not}}, year = 2013 }
• 
  Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013) S. Kounev; (S. Kounev; S. Zschaler; K. Sachs, eds.) (2013). ACM.
  • [ BibTeX ]
  @proceedings{KoZsSa2013-HOTTOPICS-Proceedings, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Zschaler, Steffen and Sachs, Kai}, keywords = {t_proceedings}, month = {04}, publisher = {ACM}, title = {Proceedings of the 2013 International Workshop on Hot Topics in Cloud Services (HotTopiCS 2013)}, year = 2013 }
• 
  Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning. N. R. Herbst; N. Huber; S. Kounev; E. Amrehn; in Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013) (2013). 187–198.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.

  @inproceedings{HeHuKoAm2013-ICPE-WorkloadClassificationAndForecasting, abstract = {{As modern enterprise software systems become increasingly dynamic, workload forecasting techniques are gaining in importance as a foundation for online capacity planning and resource management. Time series analysis covers a broad spectrum of methods to calculate workload forecasts based on history monitoring data. Related work in the field of workload forecasting mostly concentrates on evaluating specific methods and their individual optimisation potential or on predicting Quality-of-Service (QoS) metrics directly. As a basis, we present a survey on established forecasting methods of the time series analysis concerning their benefits and drawbacks and group them according to their computational overheads. In this paper, we propose a novel self-adaptive approach that selects suitable forecasting methods for a given context based on a decision tree and direct feedback cycles together with a corresponding implementation. The user needs to provide only his general forecasting objectives. In several experiments and case studies based on real world workload traces, we show that our implementation of the approach provides continuous and reliable forecast results at run-time. The results of this extensive evaluation show that the relative error of the individual forecast points is significantly reduced compared to statically applied forecasting methods, e.g. in an exemplary scenario on average by 37%. In a case study, between 55% and 75% of the violations of a given service level agreement can be prevented by applying proactive resource provisioning based on the forecast results of our implementation.}}, address = {New York, NY, USA}, author = {Herbst, Nikolas Roman and Huber, Nikolaus and Kounev, Samuel and Amrehn, Erich}, booktitle = {Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013)}, keywords = {Self-adaptive-systems}, month = {04}, pages = {187–198}, publisher = {ACM}, title = {{Self-Adaptive Workload Classification and Forecasting for Proactive Resource Provisioning}}, year = 2013 }
• 
  A Meta-Model for Performance Modeling of Dynamic Virtualized Network Infrastructures. P. Rygielski; S. Zschaler; S. Kounev; in Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013) (2013). 327–330.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this work-in-progress paper, we present a new meta-model designed for the performance modeling of dynamic data center network infrastructures. Our approach models characteristic aspects of Cloud data centers which were not crucial in classical data centers. We present our meta-model and demonstrate its use for performance modeling and analysis through an example, including a transformation into OMNeT++ for performance simulation.

  @inproceedings{RyZsKo2013-DNI-meta-model, abstract = {In this work-in-progress paper, we present a new meta-model designed for the performance modeling of dynamic data center network infrastructures. Our approach models characteristic aspects of Cloud data centers which were not crucial in classical data centers. We present our meta-model and demonstrate its use for performance modeling and analysis through an example, including a transformation into OMNeT++ for performance simulation.}, address = {New York, NY, USA}, author = {Rygielski, Piotr and Zschaler, Steffen and Kounev, Samuel}, booktitle = {Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013)}, keywords = {DNI}, month = {04}, pages = {327–330}, publisher = {ACM}, title = {{A Meta-Model for Performance Modeling of Dynamic Virtualized Network Infrastructures}}, year = 2013 }
• 
  Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions. S. Kounev; C. Rathfelder; B. Klatt; in Electronic Notes in Theoretical Computer Science (ENTCS) (2013). 295 3–9.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.

  @article{KoRaKl2012-FESCA-Keynote, abstract = {Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. Such systems typically have stringent requirements for performance and scalability as they provide business and mission critical services. While the use of event-based communication enables loosely-coupled interactions between components and leads to improved system scalability, it makes it much harder for developers to estimate the system's behavior and performance under load due to the decoupling of components and control flow. We present an overview on our approach enabling the modeling and performance prediction of event-based system at the architecture level. Applying a model-to-model transformation, our approach integrates platform-specific performance influences of the underlying middleware while enabling the use of different existing analytical and simulation-based prediction techniques. The results of two real world case studies demonstrate the effectiveness, practicability and accuracy of the proposed modeling and prediction approach.}, address = {Amsterdam, The Netherlands}, author = {Kounev, Samuel and Rathfelder, Christoph and Klatt, Benjamin}, journal = {{Electronic Notes in Theoretical Computer Science (ENTCS)}}, keywords = {t_journalmagazine}, month = {05}, pages = {3–9}, publisher = {Elsevier Science Publishers B. V.}, title = {{Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions}}, volume = 295, year = 2013 }
• 
  Conceptual Approach for Performance Isolation in Multi-Tenant Systems. M. Loesch; R. Krebs; in Proceedings of the 3rd International Conference on Cloud Computing and Service Science (CLOSER 2013) (2013).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{LoKr2013-Closer-Isolation, author = {Loesch, Manuel and Krebs, Rouven}, booktitle = {{Proceedings of the 3rd International Conference on Cloud Computing and Service Science (CLOSER 2013)}}, keywords = {se2}, month = {05}, organization = {RWTH Aachen, Germany}, publisher = {SciTePress}, title = {{Conceptual Approach for Performance Isolation in Multi-Tenant Systems}}, year = 2013 }
• 
  On Benchmarking Intrusion Detection Systems in Virtualized Environments A. Milenkoski; S. Kounev; A. Avritzer; N. Antunes; M. Vieira; (2013).
  • [ Abstract ]
  • [ BibTeX ]
  Modern intrusion detection systems (IDSes) for virtualized environments are deployed in the virtualization layer with components inside the virtual machine monitor (VMM) and the trusted host virtual machine (VM). Such IDSes can monitor at the same time the network and host activities of all guest VMs running on top of a VMM being isolated from malicious users of these VMs. We refer to IDSes for virtualized environments as VMM-based IDSes. In this work, we analyze state-of-the-art intrusion detection techniques applied in virtualized environments and architectures of VMM-based IDSes. Further, we identify challenges that apply specifically to benchmarking VMM-based IDSes focussing on workloads and metrics. For example, we discuss the challenge of defining representative baseline benign workload profiles as well as the challenge of defining malicious workloads containing attacks targeted at the VMM. We also discuss the impact of on-demand resource provisioning features of virtualized environments (e.g., CPU and memory hotplugging, memory ballooning) on IDS benchmarking measures such as capacity and attack detection accuracy. Finally, we outline future research directions in the area of benchmarking VMM-based IDSes and of intrusion detection in virtualized environments in general.

  @techreport{MiKoAvAnVi2013-TechReport-OnBenchmarkingIDSes, abstract = {{Modern intrusion detection systems (IDSes) for virtualized environments are deployed in the virtualization layer with components inside the virtual machine monitor (VMM) and the trusted host virtual machine (VM). Such IDSes can monitor at the same time the network and host activities of all guest VMs running on top of a VMM being isolated from malicious users of these VMs. We refer to IDSes for virtualized environments as VMM-based IDSes. In this work, we analyze state-of-the-art intrusion detection techniques applied in virtualized environments and architectures of VMM-based IDSes. Further, we identify challenges that apply specifically to benchmarking VMM-based IDSes focussing on workloads and metrics. For example, we discuss the challenge of defining representative baseline benign workload profiles as well as the challenge of defining malicious workloads containing attacks targeted at the VMM. We also discuss the impact of on-demand resource provisioning features of virtualized environments (e.g., CPU and memory hotplugging, memory ballooning) on IDS benchmarking measures such as capacity and attack detection accuracy. Finally, we outline future research directions in the area of benchmarking VMM-based IDSes and of intrusion detection in virtualized environments in general.}}, address = {{7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}}, author = {Milenkoski, Aleksandar and Kounev, Samuel and Avritzer, Alberto and Antunes, Nuno and Vieira, Marco}, institution = {SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC)}, keywords = {Reliability}, month = {06}, title = {{On Benchmarking Intrusion Detection Systems in Virtualized Environments}}, type = {{Technical Report SPEC-RG-2013-002 v.1.0}}, year = 2013 }
• 
  SPEC Research Group Newsletter, vol. 1 no. 2. S. Kounev; K. Sachs; P. Rygielski; (2013, June).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-2, author = {Kounev, Samuel and Sachs, Kai and Rygielski, Piotr}, keywords = {SPEC}, month = {06}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 1 no. 2}}, year = 2013 }

• 
  Interval arithmetic using expression templates, template meta programming and the upcoming C++ standard. M. Nehmeier; in Computing (2012). 94(2-4) 215–228.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In this paper we will discuss different realizations for an efficient interval arithmetic implementation using expression templates and template meta programming in C++. We will improve the handling of the rounding mode switches using expression templates and show how the constructed expression trees can be combined with other features like automatic differentiation. For a further improvement of the run time performance we try to move as many functionality as possible to the compile time using template meta programming techniques. In addition we will illustrate how an interval arithmetic implementation will profit from new features and keywords defined in the upcoming C++ standard.

  @article{nehmeierScan2010, abstract = {In this paper we will discuss different realizations for an efficient interval arithmetic implementation using expression templates and template meta programming in C++. We will improve the handling of the rounding mode switches using expression templates and show how the constructed expression trees can be combined with other features like automatic differentiation. For a further improvement of the run time performance we try to move as many functionality as possible to the compile time using template meta programming techniques. In addition we will illustrate how an interval arithmetic implementation will profit from new features and keywords defined in the upcoming C++ standard.}, author = {Nehmeier, Marco}, journal = {Computing}, keywords = {Reliability}, number = {2-4}, pages = {215-228}, publisher = {Springer Vienna}, title = {Interval arithmetic using expression templates, template meta programming and the upcoming {C}++ standard}, volume = 94, year = 2012 }
• 
  Architectural Concerns in Multi-Tenant SaaS Applications. R. Krebs; C. Momm; S. Kounev; in Proceedings of the 2nd International Conference on Cloud Computing and Services Science (CLOSER 2012) (2012).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KrMoKo2012-closer-multitenant-saas, author = {Krebs, Rouven and Momm, Christof and Kounev, Samuel}, booktitle = {{Proceedings of the 2nd International Conference on Cloud Computing and Services Science (CLOSER 2012)}}, keywords = {Application_quality_of_service_management}, month = {04}, publisher = {SciTePress}, title = {{Architectural Concerns in Multi-Tenant SaaS Applications}}, year = 2012 }
• 
  Systematic adoption of genetic programming for deriving software performance curves. M. Faber; J. Happe; in Proceedings of 3rd ACM/SPEC Internatioanl Conference on Performance Engineering (ICPE 2012) (2012). 33–44.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{FaHa2012-ICPE, address = {New York, NY, USA}, author = {Faber, Michael and Happe, Jens}, booktitle = {Proceedings of 3rd ACM/SPEC Internatioanl Conference on Performance Engineering (ICPE 2012)}, keywords = {se2}, month = {04}, pages = {33–44}, publisher = {ACM}, title = {Systematic adoption of genetic programming for deriving software performance curves}, year = 2012 }
• 
  Introduction to Queueing Petri Nets: Modeling Formalism, Tool Support and Case Studies. S. Kounev; (2012, April).
  • [ BibTeX ]
  @misc{talk-Kounev-AMCSPEC12, author = {Kounev, Samuel}, howpublished = {Tutorial Presentation, 3rd ACM/SPEC International Conference on Performance Engineering (ICPE 2012), Boston, USA}, keywords = {Talks}, month = {04}, title = {{Introduction to Queueing Petri Nets: Modeling Formalism, Tool Support and Case Studies}}, year = 2012 }
• 
  Introduction to Queueing Petri Nets: Modeling Formalism, Tool Support and Case Studies (Tutorial Paper). S. Kounev; S. Spinner; P. Meier; in Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering (ICPE 2012), Boston, USA, 22-25 April 2012 (2012). 9–18.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and evaluating their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. This tutorial presents an introduction to queueing Petri nets first introducing the modeling formalism itself and then summarizing the results of several modeling case studies which demonstrate how queueing Petri nets can be used for performance modeling and analysis. As part of the tutorial, we present QPME (Queueing Petri net Modeling Environment), an open-source tool for stochastic modeling and analysis of systems using queueing Petri nets. Finally, we briefly present a model-to-model transformation automatically generating a queueing Petri net model from a higher-level software architecture model annotated with performance relevant information.

  @inproceedings{KoSpMe2012-icpe-QPME_Tutorial, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and evaluating their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. This tutorial presents an introduction to queueing Petri nets first introducing the modeling formalism itself and then summarizing the results of several modeling case studies which demonstrate how queueing Petri nets can be used for performance modeling and analysis. As part of the tutorial, we present QPME (Queueing Petri net Modeling Environment), an open-source tool for stochastic modeling and analysis of systems using queueing Petri nets. Finally, we briefly present a model-to-model transformation automatically generating a queueing Petri net model from a higher-level software architecture model annotated with performance relevant information.}, address = {New York, NY, USA}, author = {Kounev, Samuel and Spinner, Simon and Meier, Philipp}, booktitle = {Proceedings of the 3rd ACM/SPEC International Conference on Performance Engineering (ICPE 2012), Boston, USA, 22-25 April 2012}, keywords = {QPN}, month = {04}, pages = {9–18}, publisher = {ACM}, title = {Introduction to Queueing Petri Nets: Modeling Formalism, Tool Support and Case Studies (Tutorial Paper)}, year = 2012 }
• 
  Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions. S. Kounev; (2012, March).
  • [ BibTeX ]
  @misc{talk-Kounev-FESCA12, author = {Kounev, Samuel}, howpublished = {Keynote Talk, 9th International Workshop on Formal Engineering approaches to Software Components and Architectures (FESCA @ ETAPS 2012), Tallinn, Estonia}, keywords = {Talks}, month = {03}, title = {{Modeling of Event-based Communication in Component-based Architectures: State-of-the-Art and Future Directions}}, year = 2012 }
• 
  Industry Trends and Research Challenges in Cloud Computing. S. Kounev; (2012, March).
  • [ BibTeX ]
  @misc{talk-Kounev-MultiSkill-KIT12, author = {Kounev, Samuel}, howpublished = {Keynote Talk, Multi-Skills Training Camp, "RELATE" Marie Curie Initial Training Network (ITN), Karlsruhe, Germany}, keywords = {Talks}, month = {03}, title = {{Industry Trends and Research Challenges in Cloud Computing}}, year = 2012 }
• 
  A Framework for Multi-level SLA Management. W. Theilmann; S. G. Gomez; J. Kennedy; D. Lorenzoli; C. Rathfelder; T. Roeblitz; G. Zacco; in Handbook of Research on Service-Oriented Systems and Non-Functional Properties: Future Directions, S. Reiff-Marganiec, M. Tilly (eds.) (2012). 470–490.
  • [ BibTeX ]
  • [ URL ]
  @incollection{theilmann2010a, address = {Hershey, PA, USA}, author = {Theilmann, Wolfgang and Gomez, Sergio Garcia and Kennedy, John and Lorenzoli, Davide and Rathfelder, Christoph and Roeblitz, Thomas and Zacco, Gabriele}, booktitle = {{Handbook of Research on Service-Oriented Systems and Non-Functional Properties: Future Directions}}, editor = {Reiff-Marganiec, Stephan and Tilly, Marcel}, keywords = {se2}, pages = {470–490}, publisher = {IGI Global}, title = {{A} {F}ramework for {M}ulti-level {SLA} {M}anagement}, year = 2012 }
• 
  Expression Templates and OpenCL. U. Bawidamann; M. Nehmeier; in Parallel Processing and Applied Mathematics, R. Wyrzykowski, J. Dongarra, K. Karczewski, J. Wa’s}}niewski (eds.) (2012). (Vol. 7204) 71–80.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this paper we discuss the interaction of expression templates with OpenCL devices. We show how the expression tree of expression templates can be used to generate problem specific OpenCL kernels. In a second approach we use expression templates to optimize the data transfer between the host and the device which leads to a measurable performance increase in a domain specific language approach. We tested the functionality, correctness and performance for both implementations in a case study for vector and matrix operations.

  @incollection{nehmeierPPAM2011, abstract = {In this paper we discuss the interaction of expression templates with OpenCL devices. We show how the expression tree of expression templates can be used to generate problem specific OpenCL kernels. In a second approach we use expression templates to optimize the data transfer between the host and the device which leads to a measurable performance increase in a domain specific language approach. We tested the functionality, correctness and performance for both implementations in a case study for vector and matrix operations.}, author = {Bawidamann, Uwe and Nehmeier, Marco}, booktitle = {Parallel Processing and Applied Mathematics}, editor = {Wyrzykowski, Roman and Dongarra, Jack and Karczewski, Konrad and Wa{\'s}niewski, Jerzy}, keywords = {Reliability}, pages = {71-80}, publisher = {Springer Berlin Heidelberg}, series = {Lecture Notes in Computer Science}, title = {{E}xpression {T}emplates and {OpenCL}}, volume = 7204, year = 2012 }
• 
  Online prediction: Four case studies. K. Gilly; F. Brosig; R. Nou; S. Kounev; C. Juiz; in Resilience Assessment and Evaluation of Computing Systems, K. Wolter, A. Avritzer, M. Vieira, A. van Moorsel (eds.) (2012).

  ISBN: 978-3-642-29031-2
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Current computing systems are becoming increasingly complex in nature and exhibit large variations in workloads. These changing environments create challenges to the design of systems that can adapt themselves while maintaining desired Quality of Service (QoS), security, dependability, availability and other non-functional requirements. The next generation of resilient systems will be highly distributed, component-based and service-oriented. They will need to operate in unattended mode and possibly in hostile environments, will be composed of a large number of interchangeable components discoverable at run-time, and will have to run on a multitude of unknown and heterogeneous hardware and network platforms. These computer systems will adapt themselves to cope with changes in the operating conditions and to meet the service-level agreements with a minimum of resources. Changes in operating conditions include hardware and software failures, load variation and variations in user interaction with the system, including security attacks and overwhelming situations. This self adaptation of next resilient systems can be achieved by first online predicting how these situations would be by observation of the current environment. This chapter focuses on the use of online predicting methods, techniques and tools for resilient systems. Thus, we survey online QoS adaptive models in several environments as grid environments, service-oriented architectures and ambient intelligence using different approaches based on queueing networks, model checking, ontology engineering among others.

  @incollection{GiBrNoKoJu2012-ResBook-OnlinePrediction, abstract = {{Current computing systems are becoming increasingly complex in nature and exhibit large variations in workloads. These changing environments create challenges to the design of systems that can adapt themselves while maintaining desired Quality of Service (QoS), security, dependability, availability and other non-functional requirements. The next generation of resilient systems will be highly distributed, component-based and service-oriented. They will need to operate in unattended mode and possibly in hostile environments, will be composed of a large number of interchangeable components discoverable at run-time, and will have to run on a multitude of unknown and heterogeneous hardware and network platforms. These computer systems will adapt themselves to cope with changes in the operating conditions and to meet the service-level agreements with a minimum of resources. Changes in operating conditions include hardware and software failures, load variation and variations in user interaction with the system, including security attacks and overwhelming situations. This self adaptation of next resilient systems can be achieved by first online predicting how these situations would be by observation of the current environment. This chapter focuses on the use of online predicting methods, techniques and tools for resilient systems. Thus, we survey online QoS adaptive models in several environments as grid environments, service-oriented architectures and ambient intelligence using different approaches based on queueing networks, model checking, ontology engineering among others.}}, address = {Berlin, Heidelberg}, author = {Gilly, Katja and Brosig, Fabian and Nou, Ramon and Kounev, Samuel and Juiz, Carlos}, booktitle = {Resilience Assessment and Evaluation of Computing Systems}, editor = {Wolter, K. and Avritzer, A. and Vieira, M. and van Moorsel, A.}, keywords = {Reliability}, note = {ISBN: 978-3-642-29031-2}, publisher = {Springer-Verlag}, series = {XVIII}, title = {Online prediction: Four case studies}, year = 2012 }
• 
  Generative Programming for Automatic Differentiation. M. Nehmeier; in Recent Advances in Algorithmic Differentiation, S. Forth, P. Hovland, E. Phipps, J. Utke, A. Walther (eds.) (2012). (Vol. 87) 261–271.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In this paper we present a concept for a C++ implementation of forward automatic differentiation of an arbitrary order using expression templates and template metaprogramming. In contrast to other expression template implementations, the expression tree in our implementation has only symbolic characteristics. The run-time code is then generated from the tree structure using template metaprogramming functions to apply the rules of symbolic differentiation onto the single operations at compile-time. This generic approach has the advantage that the template metaprogramming functions are replaceable which offers the opportunity to easily generate different specialized algorithms. We tested the functionality, correctness and performance of a prototype in different case studies for floating point as well as interval data types and compared it against other implementations.

  @incollection{nehmeierAD2012, abstract = {In this paper we present a concept for a C++ implementation of forward automatic differentiation of an arbitrary order using expression templates and template metaprogramming. In contrast to other expression template implementations, the expression tree in our implementation has only symbolic characteristics. The run-time code is then generated from the tree structure using template metaprogramming functions to apply the rules of symbolic differentiation onto the single operations at compile-time. This generic approach has the advantage that the template metaprogramming functions are replaceable which offers the opportunity to easily generate different specialized algorithms. We tested the functionality, correctness and performance of a prototype in different case studies for floating point as well as interval data types and compared it against other implementations.}, author = {Nehmeier, Marco}, booktitle = {Recent Advances in Algorithmic Differentiation}, editor = {Forth, Shaun and Hovland, Paul and Phipps, Eric and Utke, Jean and Walther, Andrea}, keywords = {Reliability}, pages = {261-271}, publisher = {Springer Berlin Heidelberg}, series = {Lecture Notes in Computational Science and Engineering}, title = {{G}enerative {P}rogramming for {A}utomatic {D}ifferentiation}, volume = 87, year = 2012 }
• 
  Cloud Computing Challenges and Opportunities for Service Providers. S. Kounev; (2012, May).
  • [ BibTeX ]
  @misc{talk-Kounev-Cloudzone12, author = {Kounev, Samuel}, howpublished = {Invited Talk, CLOUDZONE 2012, Karlsruher Messe- und Kongress-GmbH, Karlsruhe, Germany}, keywords = {Talks}, month = {05}, title = {{Cloud Computing Challenges and Opportunities for Service Providers}}, year = 2012 }
• 
  Specification of hardware for interval arithmetic. M. Nehmeier; S. Siegel; J. Wolff von Gudenberg; in Computing (2012). 94(2-4) 243–255.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Interval arithmetic, as it is standardized by the IEEE working group P1788 can be implemented by using floating point arithmetic units with directed rounding modes. The easiest way to represent an interval is by its two bounds. Simple formulas for the arithmetic operations can be applied. Our goal is to perform interval operations as fast as their floating point counterparts. Hence, we provide at least two units per operation. We also specify the operation for reverse multiplication (Neumaier in Vienna proposal for interval standardization, 2008) which can be implemented with the division unit. In this paper we do not care about optimization. Our primary intention is to give an easily understandable specification of hardware for interval arithmetic.

  @article{iaHardwareScan2010, abstract = {Interval arithmetic, as it is standardized by the IEEE working group P1788 can be implemented by using floating point arithmetic units with directed rounding modes. The easiest way to represent an interval is by its two bounds. Simple formulas for the arithmetic operations can be applied. Our goal is to perform interval operations as fast as their floating point counterparts. Hence, we provide at least two units per operation. We also specify the operation for reverse multiplication (Neumaier in Vienna proposal for interval standardization, 2008) which can be implemented with the division unit. In this paper we do not care about optimization. Our primary intention is to give an easily understandable specification of hardware for interval arithmetic.}, author = {Nehmeier, Marco and Siegel, Stefan and Wolff von Gudenberg, J{\"u}rgen}, journal = {Computing}, keywords = {Reliability}, number = {2-4}, pages = {243-255}, publisher = {Springer Vienna}, title = {Specification of hardware for interval arithmetic}, volume = 94, year = 2012 }
• 
  A Method for Experimental Analysis and Modeling of Virtualization Performance Overhead. N. Huber; M. von Quast; F. Brosig; M. Hauck; S. Kounev; in Cloud Computing and Services Science, I. Ivanov, M. van Sinderen, B. Shishkov (eds.) (2012). 353–370.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @incollection{HuQuBrHaKo2012-CCaSS-ExpVirtPerfOverhead, address = {New York}, author = {Huber, Nikolaus and von Quast, Marcel and Brosig, Fabian and Hauck, Michael and Kounev, Samuel}, booktitle = {Cloud Computing and Services Science}, editor = {Ivanov, Ivan and van Sinderen, Marten and Shishkov, Boris}, keywords = {t_bookchapter}, pages = {353–370}, publisher = {Springer}, series = {Service Science: Research and Innovations in the Service Economy}, title = {{A Method for Experimental Analysis and Modeling of Virtualization Performance Overhead}}, year = 2012 }
• 
  Providing Dependability and Resilience in the Cloud: Challenges and Opportunities. S. Kounev; P. Reinecke; F. Brosig; J. T. Bradley; K. Joshi; V. Babka; A. Stefanek; S. Gilmore; in Resilience Assessment and Evaluation of Computing Systems, K. Wolter, A. Avritzer, M. Vieira, A. van Moorsel (eds.) (2012).

  ISBN: 978-3-642-29031-2
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Cloud Computing is a novel paradigm for providing data center resources as on demand services in a pay-as-you-go manner. It promises significant cost savings by making it possible to consolidate workloads and share infrastructure resources among multiple applications resulting in higher cost- and energy-efficiency. However, these benefits come at the cost of increased system complexity and dynamicity posing new challenges in providing service dependability and resilience for applications running in a Cloud environment. At the same time, the virtualization of physical resources, inherent in Cloud Computing, provides new opportunities for novel dependability and quality-of-service management techniques that can potentially improve system resilience. In this chapter, we first discuss in detail the challenges and opportunities introduced by the Cloud Computing paradigm. We then provide a review of the state-of-the-art on dependability and resilience management in Cloud environments, and conclude with an overview of emerging research directions.

  @incollection{KoReBrBrJoBaStGi2012-ResBook-CloudChallenges, abstract = {{Cloud Computing is a novel paradigm for providing data center resources as on demand services in a pay-as-you-go manner. It promises significant cost savings by making it possible to consolidate workloads and share infrastructure resources among multiple applications resulting in higher cost- and energy-efficiency. However, these benefits come at the cost of increased system complexity and dynamicity posing new challenges in providing service dependability and resilience for applications running in a Cloud environment. At the same time, the virtualization of physical resources, inherent in Cloud Computing, provides new opportunities for novel dependability and quality-of-service management techniques that can potentially improve system resilience. In this chapter, we first discuss in detail the challenges and opportunities introduced by the Cloud Computing paradigm. We then provide a review of the state-of-the-art on dependability and resilience management in Cloud environments, and conclude with an overview of emerging research directions.}}, address = {Berlin, Heidelberg}, author = {Kounev, Samuel and Reinecke, Philipp and Brosig, Fabian and Bradley, Jeremy T. and Joshi, Kaustubh and Babka, Vlastimil and Stefanek, Anton and Gilmore, Stephen}, booktitle = {Resilience Assessment and Evaluation of Computing Systems}, editor = {Wolter, K. and Avritzer, A. and Vieira, M. and van Moorsel, A.}, keywords = {Power}, note = {ISBN: 978-3-642-29031-2}, publisher = {Springer-Verlag}, series = {XVIII}, title = {Providing Dependability and Resilience in the Cloud: Challenges and Opportunities}, year = 2012 }
• 
  Parallel Detection of Interval Overlapping. M. Nehmeier; S. Siegel; J. Wolff von Gudenberg; in Applied Parallel and Scientific Computing, K. Jönasson (ed.) (2012). (Vol. 7134) 127–136.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this paper we define the interval overlapping relation and develop a parallel hardware unit for its realization. As one application we consider the interval comparisons. It is shown that a detailed classification of the interval overlapping relation leads to a reduction of floating-point comparisons in common applications.

  @incollection{nehmeierPara2010, abstract = {In this paper we define the interval overlapping relation and develop a parallel hardware unit for its realization. As one application we consider the interval comparisons. It is shown that a detailed classification of the interval overlapping relation leads to a reduction of floating-point comparisons in common applications.}, author = {Nehmeier, Marco and Siegel, Stefan and Wolff von Gudenberg, J{\"u}rgen}, booktitle = {Applied Parallel and Scientific Computing}, editor = {J{\"o}nasson, Kristj{\"a}n}, keywords = {Reliability}, pages = {127-136}, publisher = {Springer Berlin Heidelberg}, series = {Lecture Notes in Computer Science}, title = {{P}arallel {D}etection of {I}nterval {O}verlapping}, volume = 7134, year = 2012 }
• 
  Providing Dependability and Performance in the Cloud: Case Studies. N. Huber; F. Brosig; N. Dingle; K. Joshi; S. Kounev; in Resilience Assessment and Evaluation of Computing Systems, K. Wolter, A. Avritzer, M. Vieira, A. van Moorsel (eds.) (2012).

  ISBN: 978-3-642-29031-2
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @incollection{HuBrDiJoKo2012-ResBook-CloudCaseStudies, address = {Berlin, Heidelberg}, author = {Huber, Nikolaus and Brosig, Fabian and Dingle, N. and Joshi, K. and Kounev, Samuel}, booktitle = {{Resilience Assessment and Evaluation of Computing Systems}}, editor = {Wolter, K. and Avritzer, A. and Vieira, M. and van Moorsel, A.}, keywords = {Optimization}, note = {ISBN: 978-3-642-29031-2}, publisher = {Springer-Verlag}, series = {XVIII}, title = {{Providing Dependability and Performance in the Cloud: Case Studies}}, year = 2012 }
• 
  Resilience Benchmarking. M. Vieira; H. Madeira; K. Sachs; S. Kounev; in Resilience Assessment and Evaluation of Computing Systems, K. Wolter, A. Avritzer, M. Vieira, A. van Moorsel (eds.) (2012).

  ISBN: 978-3-642-29031-2
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @incollection{ViMaSaKo2012-ResBook-ResilBenchmark, address = {Berlin, Heidelberg}, author = {Vieira, Marco and Madeira, Henrique and Sachs, Kai and Kounev, Samuel}, booktitle = {{Resilience Assessment and Evaluation of Computing Systems}}, editor = {Wolter, K. and Avritzer, A. and Vieira, M. and van Moorsel, A.}, keywords = {Reliability}, note = {ISBN: 978-3-642-29031-2}, publisher = {Springer-Verlag}, series = {XVIII}, title = {{Resilience Benchmarking}}, year = 2012 }
• 
  Model-based Techniques for Performance Engineering of Business Information Systems. S. Kounev; N. Huber; S. Spinner; F. Brosig; in Business Modeling and Software Design, B. Shishkov (ed.) (2012). (Vol. 0109) 19–37.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  With the increasing adoption of virtualization and the transition towards Cloud Computing platforms, modern business information systems are becoming increasingly complex and dynamic. This raises the challenge of guaranteeing system performance and scalability while at the same time ensuring efficient resource usage. In this paper, we present a historical perspective on the evolution of model-based performance engineering techniques for business information systems focusing on the major developments over the past several decades that have shaped the field. We survey the state-of-the-art on performance modeling and management approaches discussing the ongoing efforts in the community to increasingly bridge the gap between high-level business services and low level performance models. Finally, we wrap up with an outlook on the emergence of self-aware systems engineering as a new research area at the intersection of several computer science disciplines.

  @incollection{KoHuSpBr2012-business-inf-sys, abstract = {{With the increasing adoption of virtualization and the transition towards Cloud Computing platforms, modern business information systems are becoming increasingly complex and dynamic. This raises the challenge of guaranteeing system performance and scalability while at the same time ensuring efficient resource usage. In this paper, we present a historical perspective on the evolution of model-based performance engineering techniques for business information systems focusing on the major developments over the past several decades that have shaped the field. We survey the state-of-the-art on performance modeling and management approaches discussing the ongoing efforts in the community to increasingly bridge the gap between high-level business services and low level performance models. Finally, we wrap up with an outlook on the emergence of self-aware systems engineering as a new research area at the intersection of several computer science disciplines.}}, address = {Berlin, Heidelberg}, author = {Kounev, Samuel and Huber, Nikolaus and Spinner, Simon and Brosig, Fabian}, booktitle = {Business Modeling and Software Design}, editor = {Shishkov, Boris}, keywords = {t_bookchapter}, pages = {19–37}, publisher = {Springer-Verlag}, series = {Lecture Notes in Business Information Processing (LNBIP)}, title = {Model-based Techniques for Performance Engineering of Business Information Systems}, volume = {0109}, year = 2012 }
• 
  Model-based Self-Adaptive Resource Management in Virtualized Environments. S. Kounev; (2012, April).
  • [ BibTeX ]
  @misc{talk-Kounev-IBM12, author = {Kounev, Samuel}, howpublished = {Invited Talk, IBM T.J. Watson Research Center, New York, USA}, keywords = {Talks}, month = {04}, title = {{Model-based Self-Adaptive Resource Management in Virtualized Environments}}, year = 2012 }
• 
  SPEC Research Group Newsletter, vol. 1 no. 1. S. Kounev; K. Sachs; P. Rygielski; (2012, September).

  Published by Standard Performance Evaluation Corporation (SPEC)
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @misc{SPEC-RG-NEWSLETTER-1, author = {Kounev, Samuel and Sachs, Kai and Rygielski, Piotr}, keywords = {SPEC}, month = {09}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 1 no. 1}}, year = 2012 }
• 
  Engineering of Self-Aware Software Systems -- An Online Model-based Approach. S. Kounev; (2012, May).
  • [ BibTeX ]
  @misc{talk-Kounev-Dagstuhl12, author = {Kounev, Samuel}, howpublished = {Dagstuhl Seminar 12211 ''Quality-of-Service Attributes in Service- and Cloud-based Systems'', Schloss Dagstuhl, Germany}, keywords = {Talks}, month = {05}, title = {{Engineering of Self-Aware Software Systems – An Online Model-based Approach}}, year = 2012 }
• 
  Workload-aware System Monitoring Using Performance Predictions Applied to a Large-scale E-Mail System. C. Rathfelder; S. Becker; K. Krogmann; R. Reussner; in Proceedings of the Joint 10th Working IEEE/IFIP Conference on Software Architecture (WICSA) & 6th European Conference on Software Architecture (ECSA) (2012). 31–40.

  Acceptance Rate (Full Paper): 19.8%
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{rathfelder2012a, author = {Rathfelder, Christoph and Becker, Stefan and Krogmann, Klaus and Reussner, Ralf}, booktitle = {Proceedings of the Joint 10th Working IEEE/IFIP Conference on Software Architecture (WICSA) & 6th European Conference on Software Architecture (ECSA)}, keywords = {se2}, month = {08}, note = {Acceptance Rate (Full Paper): 19.8\%}, pages = {31–40}, title = {Workload-aware System Monitoring Using Performance Predictions Applied to a Large-scale E-Mail System}, year = 2012 }
• 
  Towards Benchmarking Intrusion Detection Systems for Virtualized Cloud Environments. A. Milenkoski; S. Kounev; in Proceedings of the 7th International Conference for Internet Technology and Secured Transactions (ICITST 2012) (2012). 562–563.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Many recent research works propose novel architectures of intrusion detection systems specifically designed to operate in virtualized environments. However, little attention has been given to the evaluation and benchmarking of such architectures with respect to their performance and dependability. In this paper, we present a research roadmap towards developing a framework for benchmarking intrusion detection systems for cloud environments in a scientifically rigorous and a representative manner.

  @inproceedings{MiKo2011-ICITST-TowardsBenchmarking, abstract = {{Many recent research works propose novel architectures of intrusion detection systems specifically designed to operate in virtualized environments. However, little attention has been given to the evaluation and benchmarking of such architectures with respect to their performance and dependability. In this paper, we present a research roadmap towards developing a framework for benchmarking intrusion detection systems for cloud environments in a scientifically rigorous and a representative manner.}}, address = {New York, USA}, author = {Milenkoski, Aleksandar and Kounev, Samuel}, booktitle = {Proceedings of the 7th International Conference for Internet Technology and Secured Transactions (ICITST 2012)}, keywords = {Virtualization}, month = 12, pages = {562–563}, publisher = {IEEE}, title = {{Towards Benchmarking Intrusion Detection Systems for Virtualized Cloud Environments}}, year = 2012 }
• 
  Intelligentes Mapping für Indoor-Quadrocopter. N. Schmitt; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2012, December).
  • [ BibTeX ]
  @mastersthesis{Schmitt2012, address = {Am Hubland, Informatikgeb{\"a}ude, 97074 W{\"u}rzburg, Germany}, author = {Schmitt, Norbert}, keywords = {se2}, month = 12, school = {University of W{\"u}rzburg}, title = {{Intelligentes Mapping f{\"u}r Indoor-Quadrocopter}}, type = {{Bachelor Thesis}}, year = 2012 }
• 
  Entwicklung und Evaluierung eines Systems zur Bestimmung der Orientierung und Position eines Objektes durch inertiale und magnetische Sensoren. L. Iffländer; N. Gageik; Thesis; University of Würzburg. (2012, December).
  • [ BibTeX ]
  • [ Download ]
  @mastersthesis{ifflanderentwicklung, author = {Iffl{\"a}nder, Lukas and Gageik, Nils}, keywords = {se2}, month = 12, school = {University of W{\"u}rzburg}, title = {{Entwicklung und Evaluierung eines Systems zur Bestimmung der Orientierung und Position eines Objektes durch inertiale und magnetische Sensoren}}, type = {Bachelor Thesis}, year = 2012 }
• 
  Design-Time vs. Run-Time Models for Quality-of-Service Prediction. S. Kounev; (2012, December).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-relate-KIT-12, author = {Kounev, Samuel}, howpublished = {Keynote Talk, Open Excellence Workshop on "Online Models for Quality-of-Service and Resource Management in Modern IT Systems" organized by the "RELATE" FP7 Marie Curie Initial Training Network (ITN), Karlsruhe, Germany}, keywords = {Talks}, month = 12, title = {{Design-Time vs. Run-Time Models for Quality-of-Service Prediction}}, year = 2012 }
• 
  Ingenieurmässige Entwicklung verlässlicher und effizienter Software-Systeme. S. Kounev; (2012, November).
  • [ BibTeX ]
  @misc{talk-Kounev-InvUlm12, author = {Kounev, Samuel}, howpublished = {Invited Talk, Faculty of Engineering and Computer Science, University of Ulm, Ulm, Germany}, keywords = {Talks}, month = 11, title = {{Ingenieurm\"{a}ssige Entwicklung verl\"{a}sslicher und effizienter Software-Systeme}}, year = 2012 }
• 
  Towards Truthful Resource Reservation in Cloud Computing. D. Funke; F. Brosig; M. Faber; in Proceedings of the 6th International ICST Conference on Performance Evaluation Methodologies and Tools (ValueTools 2012), Carg`ese, France (2012).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Prudent capacity planning to meet their clients future computational needs is one of the major issues cloud computing providers face today. By offering resource reservations in advance, providers gain insight into the projected demand of their customers and can act accordingly. However, customers need to be given an incentive, e.g. discounts granted, to commit early to a provider and to honestly, i.e., truthfully reserve their predicted future resource requirements. Customers may reserve capacity deviating from their truly predicted demand, in order to exploit the mechanism for their own benefit, thereby causing futile costs for the provider. In this paper we prove, using a game theoretic approach, that truthful reservation is the best, i.e., dominant strategy for customers if they are capable to make precise forecasts of their demands and that deviations from truth-telling can be profitable for customers if their demand forecasts are uncertain.

  @inproceedings{FuBrFa2012-VALUETOOLS-TruthfulResourceReservation, abstract = {{Prudent capacity planning to meet their clients future computational needs is one of the major issues cloud computing providers face today. By offering resource reservations in advance, providers gain insight into the projected demand of their customers and can act accordingly. However, customers need to be given an incentive, e.g. discounts granted, to commit early to a provider and to honestly, i.e., truthfully reserve their predicted future resource requirements. Customers may reserve capacity deviating from their truly predicted demand, in order to exploit the mechanism for their own benefit, thereby causing futile costs for the provider. In this paper we prove, using a game theoretic approach, that truthful reservation is the best, i.e., dominant strategy for customers if they are capable to make precise forecasts of their demands and that deviations from truth-telling can be profitable for customers if their demand forecasts are uncertain.}}, author = {Funke, Daniel and Brosig, Fabian and Faber, Michael}, booktitle = {{Proceedings of the 6th International ICST Conference on Performance Evaluation Methodologies and Tools (ValueTools 2012), Carg{`e}se, France}}, keywords = {Optimization}, month = 10, title = {{Towards Truthful Resource Reservation in Cloud Computing}}, year = 2012 }
• 
  S/T/A: Meta-Modeling Run-Time Adaptation in Component-Based System Architectures. N. Huber; A. van Hoorn; A. Koziolek; F. Brosig; S. Kounev; in Proceedings of the 9th IEEE International Conference on e-Business Engineering (ICEBE 2012) (2012). 70–77.

  Acceptance Rate (Full Paper): 19.7% (26/132)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modern virtualized system environments usually host diverse applications of different parties and aim at utilizing resources efficiently while ensuring that quality-of-service requirements are continuously satisfied. In such scenarios, complex adaptations to changes in the system environment are still largely performed manually by humans. Over the past decade, autonomic self-adaptation techniques aiming to minimize human intervention have become increasingly popular. However, given that adaptation processes are usually highly system specific, it is a challenge to abstract from system details enabling the reuse of adaptation strategies. In this paper, we propose a novel modeling language (meta-model) providing means to describe system adaptation processes at the system architecture level in a generic, human-understandable and reusable way. We apply our approach to three different realistic contexts (dynamic resource allocation, software architecture optimization, and run-time adaptation planning) showing how the gap between complex manual adaptations and their autonomous execution can be closed by using a holistic model-based approach.

  @inproceedings{HuHoKoBrKo2012-ICEBE-STA, abstract = {Modern virtualized system environments usually host diverse applications of different parties and aim at utilizing resources efficiently while ensuring that quality-of-service requirements are continuously satisfied. In such scenarios, complex adaptations to changes in the system environment are still largely performed manually by humans. Over the past decade, autonomic self-adaptation techniques aiming to minimize human intervention have become increasingly popular. However, given that adaptation processes are usually highly system specific, it is a challenge to abstract from system details enabling the reuse of adaptation strategies. In this paper, we propose a novel modeling language (meta-model) providing means to describe system adaptation processes at the system architecture level in a generic, human-understandable and reusable way. We apply our approach to three different realistic contexts (dynamic resource allocation, software architecture optimization, and run-time adaptation planning) showing how the gap between complex manual adaptations and their autonomous execution can be closed by using a holistic model-based approach.}, address = {Los Alamitos, CA, USA}, author = {Huber, Nikolaus and van Hoorn, Andr{\'e} and Koziolek, Anne and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the 9th IEEE International Conference on e-Business Engineering (ICEBE 2012)}, keywords = {Optimization}, month = {09}, note = {Acceptance Rate (Full Paper): 19.7\% (26/132)}, pages = {70–77}, publisher = {IEEE Computer Society}, title = {{S/T/A: Meta-Modeling Run-Time Adaptation in Component-Based System Architectures}}, year = 2012 }
• 
  Automated Inference of Goal-oriented Performance Prediction Functions. D. Westermann; J. Happe; R. Krebs; R. Farahbod; in Proceedings of the 27th IEEE/ACM International Conference On Automated Software Engineering (ASE 2012) (2012).
  • [ BibTeX ]
  @inproceedings{westermann2012b, author = {Westermann, Dennis and Happe, Jens and Krebs, Rouven and Farahbod, Roozbeh}, booktitle = {Proceedings of the 27th IEEE/ACM International Conference On Automated Software Engineering (ASE 2012)}, keywords = {se2}, month = {09}, title = {Automated Inference of Goal-oriented Performance Prediction Functions}, year = 2012 }
• 
  The Descartes Meta-Model for Online QoS and Resource Management. S. Kounev; (2012, May).
  • [ BibTeX ]
  @misc{talk-Kounev-SAPResearch12, author = {Kounev, Samuel}, howpublished = {Invited Talk, SAP Research, Karlsruhe/Darmstadt, Germany}, keywords = {Talks}, month = {05}, title = {{The Descartes Meta-Model for Online QoS and Resource Management}}, year = 2012 }
• 
  Enhancing the Usability of Outliers by Description of Local Deviation. J. Walter; Thesis; Karlsruhe Institute of Technology (KIT); Am Fasanengarten 5, 76131 Karlsruhe, Germany. (2012, July).
  • [ BibTeX ]
  @mastersthesis{Walter2012-outlier-description, address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany}, author = {Walter, J{\"u}rgen}, keywords = {se2}, month = {07}, school = {Karlsruhe Institute of Technology (KIT)}, title = {{Enhancing the Usability of Outliers by Description of Local Deviation}}, type = {Bachelor Thesis}, year = 2012 }
• 
  Modeling Dynamic Virtualized Resource Landscapes. N. Huber; F. Brosig; S. Kounev; in Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012) (2012). 81–90.

  Acceptance Rate (Full Paper): 25.6%
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modern data centers are subject to an increasing demand for flexibility. Increased flexibility and dynamics, however, also result in a higher system complexity. This complexity carries on to run-time resource management for Quality-of-Service (QoS) enforcement, rendering design-time approaches for QoS assurance inadequate. In this paper, we present a set of novel meta-models that can be used to describe the resource landscape, the architecture and resource layers of dynamic virtualized data center infrastructures, as well as their run-time adaptation and resource management aspects. With these meta-models we introduce new modeling concepts to improve model-based run-time QoS assurance. We evaluate our meta-models by modeling a representative virtualized service infrastructure and using these model instances for run-time resource allocation. The results demonstrate the benefits of the new meta-models and show how they can be used to improve model-based system adaptation and run-time resource management in dynamic virtualized data centers.

  @inproceedings{HuBrKo2012-QoSA-ModelingVirtResLandscapes, abstract = {Modern data centers are subject to an increasing demand for flexibility. Increased flexibility and dynamics, however, also result in a higher system complexity. This complexity carries on to run-time resource management for Quality-of-Service (QoS) enforcement, rendering design-time approaches for QoS assurance inadequate. In this paper, we present a set of novel meta-models that can be used to describe the resource landscape, the architecture and resource layers of dynamic virtualized data center infrastructures, as well as their run-time adaptation and resource management aspects. With these meta-models we introduce new modeling concepts to improve model-based run-time QoS assurance. We evaluate our meta-models by modeling a representative virtualized service infrastructure and using these model instances for run-time resource allocation. The results demonstrate the benefits of the new meta-models and show how they can be used to improve model-based system adaptation and run-time resource management in dynamic virtualized data centers.}, address = {New York, NY, USA}, author = {Huber, Nikolaus and Brosig, Fabian and Kounev, Samuel}, booktitle = {Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012)}, keywords = {Self-adaptive-systems}, month = {06}, note = {Acceptance Rate (Full Paper): 25.6\%}, pages = {81–90}, publisher = {ACM}, title = {{Modeling Dynamic Virtualized Resource Landscapes}}, year = 2012 }
• 
  Metrics and Techniques for Quantifying Performance Isolation in Cloud Environments. R. Krebs; C. Momm; S. Kounev; in Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012), B. Buhnova, A. Vallecillo (eds.) (2012). 91–100.

  Acceptance Rate (Full Paper): 25.6%
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{KrMoKo2012-QoSA-QuantifyingPerfIsoMetrics, address = {New York, USA}, author = {Krebs, Rouven and Momm, Christof and Kounev, Samuel}, booktitle = {Proceedings of the 8th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2012)}, editor = {Buhnova, Barbora and Vallecillo, Antonio}, keywords = {Application_quality_of_service_management}, month = {06}, note = {Acceptance Rate (Full Paper): 25.6\%}, pages = {91–100}, publisher = {ACM Press}, title = {{M}etrics and {T}echniques for {Q}uantifying {P}erformance {I}solation in {C}loud {E}nvironments}, year = 2012 }
• 
  Introduction to the Descartes Meta-Model (DMM). S. Kounev; (2012, June).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-InvPrague12, author = {Kounev, Samuel}, howpublished = {Invited Talk, Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic}, keywords = {Talks}, month = {06}, title = {{Introduction to the Descartes Meta-Model (DMM)}}, year = 2012 }
• 
  Modeling Parameter and Context Dependencies in Online Architecture-Level Performance Models. F. Brosig; N. Huber; S. Kounev; in Proceedings of the 15th ACM SIGSOFT International Symposium on Component Based Software Engineering (CBSE 2012), June 26--28, 2012, Bertinoro, Italy (2012).

  Acceptance Rate (Full Paper): 28.5%.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modern enterprise applications have to satisfy increasingly stringent Quality-of-Service requirements. To ensure that a system meets its performance requirements, the ability to predict its performance under different configurations and workloads is essential. Architecture-level performance models describe performance-relevant aspects of software architectures and execution environments allowing to evaluate different usage profiles as well as system deployment and configuration options. However, building performance models manually requires a lot of time and effort. In this paper, we present a novel automated method for the extraction of architecture-level performance models of distributed component-based systems, based on monitoring data collected at run-time. The method is validated in a case study with the industry-standard SPECjEnterprise2010 Enterprise Java benchmark, a representative software system executed in a realistic environment. The obtained performance predictions match the measurements on the real system within an error margin of mostly 10-20 percent.

  @inproceedings{BrHuKo2012-CBSE-ParamAndContextDep, abstract = {Modern enterprise applications have to satisfy increasingly stringent Quality-of-Service requirements. To ensure that a system meets its performance requirements, the ability to predict its performance under different configurations and workloads is essential. Architecture-level performance models describe performance-relevant aspects of software architectures and execution environments allowing to evaluate different usage profiles as well as system deployment and configuration options. However, building performance models manually requires a lot of time and effort. In this paper, we present a novel automated method for the extraction of architecture-level performance models of distributed component-based systems, based on monitoring data collected at run-time. The method is validated in a case study with the industry-standard SPECjEnterprise2010 Enterprise Java benchmark, a representative software system executed in a realistic environment. The obtained performance predictions match the measurements on the real system within an error margin of mostly 10-20 percent.}, author = {Brosig, Fabian and Huber, Nikolaus and Kounev, Samuel}, booktitle = {Proceedings of the 15th ACM SIGSOFT International Symposium on Component Based Software Engineering (CBSE 2012), June 26–28, 2012, Bertinoro, Italy}, keywords = {Application_quality_of_service_management}, month = {06}, note = {Acceptance Rate (Full Paper): 28.5\%.}, title = {{Modeling Parameter and Context Dependencies in Online Architecture-Level Performance Models}}, year = 2012 }
• 
  Modeling Virtualized Cloud Data Center Networks: Do We need QoS in Networks?. P. Rygielski; (2012, June).
  • [ BibTeX ]
  @misc{talk-Rygielski-OEW2012, author = {Rygielski, Piotr}, howpublished = {Open Excellence Workshop (Invited Talk), Charles University in Prague, Czech Republic.}, keywords = {Talks}, month = {06}, title = {{Modeling Virtualized Cloud Data Center Networks: Do We need QoS in Networks?}}, year = 2012 }
• 
  Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0. S. Spinner; S. Kounev; P. Meier; in Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012), S. Haddad, L. Pomello (eds.) (2012). (Vol. 7347) 388–397.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.

  @inproceedings{SpKoMe2012-PETRINETS-QPME, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present our tool QPME (Queueing Petri net Modeling Environment) for modeling and analysis using queueing Petri nets. QPME provides an Eclipse-based editor for building queueing Petri net models and a powerful simulation engine for analyzing these models. The development of the tool started in 2003 and since then the tool has been distributed to more than 120 organizations worldwide.}, address = {Berlin, Heidelberg}, author = {Spinner, Simon and Kounev, Samuel and Meier, Philipp}, booktitle = {Proceedings of the 33rd International Conference on Application and Theory of Petri Nets and Concurrency (Petri Nets 2012)}, editor = {Haddad, Serge and Pomello, Lucia}, keywords = {QPN}, month = {06}, pages = {388–397}, publisher = {Springer-Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Stochastic Modeling and Analysis using QPME: Queueing Petri Net Modeling Environment v2.0}}, volume = 7347, year = 2012 }
• 
  User Assignment and Movement Prediction in Wireless Networks. P. ’S}}wika}tek; P. Rygielski; K. Juszczyszyn; A. Grzech; in Cybernetics and Systems: An International Journal (CBS) (2012). 43(4) 340–353.

  5-Year Impact Factor (2014): 0.968
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The article is devoted to the problem of mobility and resource management in heterogeneous wireless networks. It is assumed that in certain areas covered by multiple overlapping wireless networks there are a certain number of mobile clients that consume networks resources by use of the available communication services (e.g., voice or data transmission) delivered by network providers. Moreover, it is assumed that the continuity of communication services may be assured by the use of common handover techniques supporting client mobility (e.g., Mobile IPv6, IEEE 802.21, etc.). The task of mobility and resource management consists of making decisions concerning the moment and the network to which particular clients should be handed over in order to optimize certain quality criterion (e.g., utilization of network resources). In this article we show that gathering knowledge about client movements and prediction of their future positions may significantly improve the overall quality of the services delivered and network resource utilization.

  @article{SwRyJuGr2012-CBS, abstract = {The article is devoted to the problem of mobility and resource management in heterogeneous wireless networks. It is assumed that in certain areas covered by multiple overlapping wireless networks there are a certain number of mobile clients that consume networks resources by use of the available communication services (e.g., voice or data transmission) delivered by network providers. Moreover, it is assumed that the continuity of communication services may be assured by the use of common handover techniques supporting client mobility (e.g., Mobile IPv6, IEEE 802.21, etc.). The task of mobility and resource management consists of making decisions concerning the moment and the network to which particular clients should be handed over in order to optimize certain quality criterion (e.g., utilization of network resources). In this article we show that gathering knowledge about client movements and prediction of their future positions may significantly improve the overall quality of the services delivered and network resource utilization.}, address = {Bristol, PA, USA}, author = {{\'S}wi\k{a}tek, Pawe{l} and Rygielski, Piotr and Juszczyszyn, Krzysztof and Grzech, Adam}, journal = {{Cybernetics and Systems: An International Journal (CBS)}}, keywords = {t_journalmagazine}, month = {05}, note = {5-Year Impact Factor (2014): 0.968}, number = 4, pages = {340–353}, publisher = {Taylor & Francis, Inc.}, title = {{User Assignment and Movement Prediction in Wireless Networks}}, volume = 43, year = 2012 }
• 
  Engineering of Dependable and Efficient Self-Aware Complex Systems. S. Kounev; (2012, July).
  • [ BibTeX ]
  @misc{talk-Kounev-InvStutt12, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), Faculty of Computer Science, Electrical Engineering and Information Technology, University of Stuttgart, Stuttgart, Germany}, keywords = {Talks}, month = {07}, title = {{Engineering of Dependable and Efficient Self-Aware Complex Systems}}, year = 2012 }

• 
  ICPE’11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14--16, 2011 S. Kounev; (S. Kounev; V. Cortellessa; R. Mirandola; D. J. Lilja, eds.) (2011). ACM, New York, NY, USA.
  • [ BibTeX ]
  @proceedings{DBLP-conf-wosp-2011, address = {New York, NY, USA}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Cortellessa, Vittorio and Mirandola, Raffaela and Lilja, David J.}, keywords = {Power}, month = {03}, publisher = {ACM}, title = {ICPE'11 - 2nd Joint ACM/SPEC International Conference on Performance Engineering, Karlsruhe, Germany, March 14–16, 2011}, year = 2011 }
• 
  Self-Aware Software and Systems Engineering: A Vision and Research Roadmap. S. Kounev; (2011, February).
  • [ BibTeX ]
  @misc{talk-Kounev-Nachwuchs11, author = {Kounev, Samuel}, howpublished = {Software Engineering 2011 (SE 2011), Nachwuchswissenschaftler-Symposium, Karlsruhe}, keywords = {Talks}, month = {02}, title = {{Self-Aware Software and Systems Engineering: A Vision and Research Roadmap}}, year = 2011 }
• 
  A Qualitative Discussion of Different Approaches for Implementing Multi-Tenant SaaS Offerings. C. Momm; R. Krebs; in Proceedings of the Software Engineering 2011 -- Workshopband (ESoSyM-2011), R. Reussner, A. amd J. Pretschner (eds.) (2011). 139–150.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{MoKr2011-esosym-qualitative-discussion, address = {Bonn-Buschdorf, Germany}, author = {Momm, Christof and Krebs, Rouven}, booktitle = {Proceedings of the Software Engineering 2011 – Workshopband (ESoSyM-2011)}, editor = {Reussner, Ralf and {Pretschner, Alexander amd J{\"a}hnichen}, Stefan}, keywords = {se2}, month = {02}, organization = {Fachgruppe OOSE der Gesellschaft f{\"u}r Informatik und ihrer Arbeitskreise}, pages = {139–150}, publisher = {Bonner K{\"o}llen Verlag}, title = {{A} {Q}ualitative {D}iscussion of {D}ifferent {A}pproaches for {I}mplementing {M}ulti-{T}enant {S}aa{S} {O}fferings}, year = 2011 }
• 
  Evaluating and Modeling Virtualization Performance Overhead for Cloud Environments. N. Huber; M. von Quast; M. Hauck; S. Kounev; in Proceedings of the 1st International Conference on Cloud Computing and Services Science (CLOSER 2011) (2011). 563–573.

  Acceptance Rate: 18/164 = 10.9%, Best Paper Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Due to trends like Cloud Computing and Green IT, virtualization technologies are gaining increasing importance. They promise energy and cost savings by sharing physical resources, thus making resource usage more efficient. However, resource sharing and other factors have direct effects on system performance, which are not yet well-understood. Hence, performance prediction and performance management of services deployed in virtualized environments like public and private Clouds is a challenging task. Because of the large variety of virtualization solutions, a generic approach to predict the performance overhead of services running on virtualization platforms is highly desirable. In this paper, we present experimental results on two popular state-of-the-art virtualization platforms, Citrix XenServer 5.5 and VMware ESX 4.0, as representatives of the two major hypervisor architectures. Based on these results, we propose a basic, generic performance prediction model for the two different types of hypervisor architectures. The target is to predict the performance overhead for executing services on virtualized platforms.

  @inproceedings{HuQuHaKo2011-CLOSER-ModelVirtOverhead, abstract = {Due to trends like Cloud Computing and Green IT, virtualization technologies are gaining increasing importance. They promise energy and cost savings by sharing physical resources, thus making resource usage more efficient. However, resource sharing and other factors have direct effects on system performance, which are not yet well-understood. Hence, performance prediction and performance management of services deployed in virtualized environments like public and private Clouds is a challenging task. Because of the large variety of virtualization solutions, a generic approach to predict the performance overhead of services running on virtualization platforms is highly desirable. In this paper, we present experimental results on two popular state-of-the-art virtualization platforms, Citrix XenServer 5.5 and VMware ESX 4.0, as representatives of the two major hypervisor architectures. Based on these results, we propose a basic, generic performance prediction model for the two different types of hypervisor architectures. The target is to predict the performance overhead for executing services on virtualized platforms.}, author = {Huber, Nikolaus and von Quast, Marcel and Hauck, Michael and Kounev, Samuel}, booktitle = {Proceedings of the 1st International Conference on Cloud Computing and Services Science (CLOSER 2011)}, keywords = {Virtualization}, month = {05}, note = {Acceptance Rate: 18/164 = 10.9\%, <b>Best Paper Award</b>}, pages = {563 – 573}, publisher = {SciTePress}, title = {{E}valuating and {M}odeling {V}irtualization {P}erformance {O}verhead for {C}loud {E}nvironments}, year = 2011 }
• 
  Software Performance Analysis using Machine Learning Techniques. M. Faber; Thesis; Karlsruhe Institute of Technology (KIT); Karlsruhe, Germany. (2011, March).
  • [ BibTeX ]
  @mastersthesis{Fa2011-KIT-SPAusingML, address = {Karlsruhe, Germany}, author = {Faber, Michael}, keywords = {se2}, month = {03}, school = {Karlsruhe Institute of Technology (KIT)}, title = {{Software Performance Analysis using Machine Learning Techniques}}, year = 2011 }
• 
  Automated Simulation-Based Capacity Planning for Enterprise Data Fabrics. S. Kounev; K. Bender; F. Brosig; N. Huber; R. Okamoto; in 4th International ICST Conference on Simulation Tools and Techniques (SIMUtools 2011), March 21--25, 2011, Barcelona, Spain (2011).

  Acceptance Rate (Full Paper): 29.8% (23/77), ICST Best Paper Award
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{KoBeBrHuOk2011-SIMUTools-DataFabrics, author = {Kounev, Samuel and Bender, Konstantin and Brosig, Fabian and Huber, Nikolaus and Okamoto, Russell}, booktitle = {4th International ICST Conference on Simulation Tools and Techniques (SIMUtools 2011), March 21–25, 2011, Barcelona, Spain}, keywords = {QPME_Bibliography}, month = {03}, note = {Acceptance Rate (Full Paper): 29.8\% (23/77), <b>ICST Best Paper Award</b>}, title = {Automated Simulation-Based Capacity Planning for Enterprise Data Fabrics}, year = 2011 }
• 
  Descartes Research Group at KIT: Towards Engineering of Self-Aware Software Systems. S. Kounev; (2011, April).
  • [ BibTeX ]
  @misc{talk-Kounev-Dagstuhl11, author = {Kounev, Samuel}, howpublished = {Invited Talk, Jahrestreffen des Beirates der Universit{\"a}tsprofessoren in der Gesselschaft f{\"u}r Informatik (GIBU), Schloss Dagstuhl}, keywords = {Talks}, month = {04}, title = {{Descartes Research Group at KIT: Towards Engineering of Self-Aware Software Systems}}, year = 2011 }
• 
  Handover: a Tool for Optimization of Wireless Networks. P. ’S}}wika}tek; P. Rygielski; A. Grzech; in Proceedings of the First European Teletraffic Seminar: an initiative to strengthen collaboration on Network Design and Performance (ETS 2011), G. Danilewicz (ed.) (2011). 80–84.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The growth of mobile devices equipped with wireless network interface on the market puts researchers focus on the new area of wireless connection management. In this paper we introduce an approach of wireless network management with handovers which occurs when user is moving through the area that is covered with more than one wireless network. The task of user’s assignment to the wireless network is considered in this paper. There have been two main tasks formulated: local optimization, where user chooses the network to connect and global one, where the network management system picks users in order to maximize users satisfaction. The users have to be assigned with proper amount of communicational resources in every moment of the passing of considered geographical area. I this paper we propose three heuristic algorithms of user QoS-constrained assignment to networks. In two algorithms the user makes decision basing on his knowledge about network and his mobile device state and in the third algorithm, the networks management system decides which users to connect. The evaluation of proposed algorithms is done by means of computer simulation.

  @inproceedings{GrSwRy2011-ETS, abstract = {The growth of mobile devices equipped with wireless network interface on the market puts researchers focus on the new area of wireless connection management. In this paper we introduce an approach of wireless network management with handovers which occurs when user is moving through the area that is covered with more than one wireless network. The task of user’s assignment to the wireless network is considered in this paper. There have been two main tasks formulated: local optimization, where user chooses the network to connect and global one, where the network management system picks users in order to maximize users satisfaction. The users have to be assigned with proper amount of communicational resources in every moment of the passing of considered geographical area. I this paper we propose three heuristic algorithms of user QoS-constrained assignment to networks. In two algorithms the user makes decision basing on his knowledge about network and his mobile device state and in the third algorithm, the networks management system decides which users to connect. The evaluation of proposed algorithms is done by means of computer simulation.}, address = {Pozna{\'n}, Poland}, author = {{\'S}wi\k{a}tek, Pawel and Rygielski, Piotr and Grzech, Adam}, booktitle = {{Proceedings of the First European Teletraffic Seminar: an initiative to strengthen collaboration on Network Design and Performance (ETS 2011)}}, editor = {Danilewicz, Grzegorz}, keywords = {descartes}, month = {02}, pages = {80–84}, publisher = {ASART STUDIO Agnieszka Chwalisz}, title = {{Handover: a Tool for Optimization of Wireless Networks}}, year = 2011 }
• 
  Online Performance Prediction with Architecture-Level Performance Models. F. Brosig; in Software Engineering (Workshops) - Doctoral Symposium, February 21--25, 2011, R. Reussner, A. Pretschner, S. Jähnichen (eds.) (2011). (Vol. 184) 279–284.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Today's enterprise systems based on increasingly complex software architectures often exhibit poor performance and resource efficiency thus having high operating costs. This is due to the inability to predict at run-time the effect of changes in the system environment and adapt the system accordingly. We propose a new performance modeling approach that allows the prediction of performance and system resource utilization online during system operation. We use architecture-level performance models that capture the performance-relevant information of the software architecture, deployment, execution environment and workload. The models will be automatically maintained during operation. To derive performance predictions, we propose a tailorable model solving approach to provide flexibility in view of prediction accuracy and analysis overhead.

  @inproceedings{Br2011-SEDoktoranden-Symposium, abstract = {{Today's enterprise systems based on increasingly complex software architectures often exhibit poor performance and resource efficiency thus having high operating costs. This is due to the inability to predict at run-time the effect of changes in the system environment and adapt the system accordingly. We propose a new performance modeling approach that allows the prediction of performance and system resource utilization online during system operation. We use architecture-level performance models that capture the performance-relevant information of the software architecture, deployment, execution environment and workload. The models will be automatically maintained during operation. To derive performance predictions, we propose a tailorable model solving approach to provide flexibility in view of prediction accuracy and analysis overhead.}}, address = {Bonn, Germany}, author = {Brosig, Fabian}, booktitle = {Software Engineering (Workshops) - Doctoral Symposium, February 21–25, 2011}, editor = {Reussner, Ralf and Pretschner, Alexander and J{\"a}hnichen, Stefan}, keywords = {t_workshop}, month = {02}, pages = {279–284}, publisher = {GI}, series = {Lecture Notes in Informatics (LNI)}, title = {Online Performance Prediction with Architecture-Level Performance Models}, volume = 184, year = 2011 }
• 
  Interval Comparisons and Lattice Operations based on the Interval Overlapping Relation. M. Nehmeier; J. Wolff von Gudenberg; in Proceedings of the World Conference on Soft Computing 2011 (WConSC’11) (2011).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The interval overlapping relation defines 13 different states of the relative position of two intervals. In this paper we show that this relation can be used to define the interval comparisons and lattice operations. The shift from a boolean (binary) comparison to a comprehensive relation (13 different states) enables the user to exploit involved dependencies. We further introduce an object oriented abstract datatype to provide a userfriendly interface.

  @inproceedings{nehmeierWconSC2011, abstract = {The interval overlapping relation defines 13 different states of the relative position of two intervals. In this paper we show that this relation can be used to define the interval comparisons and lattice operations. The shift from a boolean (binary) comparison to a comprehensive relation (13 different states) enables the user to exploit involved dependencies. We further introduce an object oriented abstract datatype to provide a userfriendly interface.}, address = {San Francisco, CA, USA}, author = {Nehmeier, Marco and Wolff von Gudenberg, J{\"u}rgen}, booktitle = {Proceedings of the World Conference on Soft Computing 2011 (WConSC'11)}, keywords = {Reliability}, title = {{I}nterval {C}omparisons and {L}attice {O}perations based on the {I}nterval {O}verlapping {R}elation}, year = 2011 }
• 
  Automatic Performance Model Extraction for Run-Time Capacity Management. S. Kounev; (2011, January).
  • [ BibTeX ]
  @misc{talk-Kounev-KielKolloq11, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), Department of Computer Science, Christian-Albrechts-Universit{\"a}t zu Kiel, Kiel}, keywords = {Talks}, month = {01}, title = {{Automatic Performance Model Extraction for Run-Time Capacity Management}}, year = 2011 }
• 
  Focussing Multi-objective Software Architecture Optimization Using Quality of Service Bounds. A. Koziolek; Q. Noorshams; R. Reussner; in Models in Software Engineering, Workshops and Symposia at MODELS 2010, Oslo, Norway, October 3-8, 2010, Reports and Revised Selected Papers, J. Dingel, A. Solberg (eds.) (2011). (Vol. 6627) 384–399.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Quantitative prediction of non-functional properties, such as performance, reliability, and costs, of software architectures supports systematic software engineering. Even though there usually is a rough idea on bounds for quality of service, the exact required values may be unclear and subject to trade-offs. Designing architectures that exhibit such good trade-off between multiple quality attributes is hard. Even with a given functional design, many degrees of freedom in the software architecture (e.g. component deployment or server configuration) span a large design space. Automated approaches search the design space with multi-objective metaheuristics such as evolutionary algorithms. However, as quality prediction for a single architecture is computationally expensive, these approaches are time consuming. In this work, we enhance an automated improvement approach to take into account bounds for quality of service in order to focus the search on interesting regions of the objective space, while still allowing trade-offs after the search. We compare two different constraint handling techniques to consider the bounds. To validate our approach, we applied both techniques to an architecture model of a component-based business information system. We compared both techniques to an unbounded search in 4 scenarios. Every scenario was examined with 10 optimization runs, each investigating around 1600 architectural candidates. The results indicate that the integration of quality of service bounds during the optimization process can improve the quality of the solutions found, however, the effect depends on the scenario, i.e. the problem and the quality requirements. The best results were achieved for costs requirements: The approach was able to decrease the time needed to find good solutions in the interesting regions of the objective space by 25% on average.

  @inproceedings{KoNoRe2011-MODELS-Invited, abstract = {Quantitative prediction of non-functional properties, such as performance, reliability, and costs, of software architectures supports systematic software engineering. Even though there usually is a rough idea on bounds for quality of service, the exact required values may be unclear and subject to trade-offs. Designing architectures that exhibit such good trade-off between multiple quality attributes is hard. Even with a given functional design, many degrees of freedom in the software architecture (e.g. component deployment or server configuration) span a large design space. Automated approaches search the design space with multi-objective metaheuristics such as evolutionary algorithms. However, as quality prediction for a single architecture is computationally expensive, these approaches are time consuming. In this work, we enhance an automated improvement approach to take into account bounds for quality of service in order to focus the search on interesting regions of the objective space, while still allowing trade-offs after the search. We compare two different constraint handling techniques to consider the bounds. To validate our approach, we applied both techniques to an architecture model of a component-based business information system. We compared both techniques to an unbounded search in 4 scenarios. Every scenario was examined with 10 optimization runs, each investigating around 1600 architectural candidates. The results indicate that the integration of quality of service bounds during the optimization process can improve the quality of the solutions found, however, the effect depends on the scenario, i.e. the problem and the quality requirements. The best results were achieved for costs requirements: The approach was able to decrease the time needed to find good solutions in the interesting regions of the objective space by 25\% on average.}, author = {Koziolek, Anne and Noorshams, Qais and Reussner, Ralf}, booktitle = {{Models in Software Engineering, Workshops and Symposia at MODELS 2010, Oslo, Norway, October 3-8, 2010, Reports and Revised Selected Papers}}, editor = {Dingel, J. and Solberg, A.}, keywords = {se2}, pages = {384–399}, publisher = {Springer-Verlag Berlin Heidelberg}, series = {Lecture Notes in Computer Science}, title = {Focussing Multi-objective Software Architecture Optimization Using Quality of Service Bounds}, volume = 6627, year = 2011 }
• 
  The Open Reference Case A Reference Use Case for the SLA@SOI Framework. C. Rathfelder; B. Klatt; G. Falcone; in Service Level Agreements for Cloud Computing, P. Wieder, J. M. Butler, W. Theilmann, R. Yahyapour (eds.) (2011). 27–40.
  • [ BibTeX ]
  • [ URL ]
  @incollection{rathfelder2011b, address = {New York}, author = {Rathfelder, Christoph and Klatt, Benjamin and Falcone, Giovanni}, booktitle = {{Service Level Agreements for Cloud Computing}}, editor = {Wieder, Philipp and Butler, Joe M. and Theilmann, Wolfgang and Yahyapour, Ramin}, keywords = {t_bookchapter}, pages = {27–40}, publisher = {Springer}, title = {The Open Reference Case A Reference Use Case for the SLA@SOI Framework}, year = 2011 }
• 
  Self-Aware Software and Systems Engineering: A Vision and Research Roadmap. S. Kounev; in GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928 (2011).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{Ko2011-SE-DescartesResearch, address = {Karlsruhe, Germany}, author = {Kounev, Samuel}, booktitle = {{GI Softwaretechnik-Trends, 31(4), November 2011, ISSN 0720-8928}}, keywords = {Self-adaptive-systems}, title = {{Self-Aware Software and Systems Engineering: A Vision and Research Roadmap}}, year = 2011 }
• 
  filib++, Expression Templates and the Coming Interval Standard. M. Nehmeier; J. Wolff von Gudenberg; in Reliable Computing (2011). 15(4) 312–320.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  In this paper we investigate how a C++ class library can be improved by the concept of expression templates. Our first result is a saving of rounding mode switches which considerably increases the performance. Our second result deals with handling the discontinuity flag that will probably be decided to be raised whenever a function is called outside its domain (loose evaluation). We discuss several alternatives and propose an expression related flag that can be used in a thread safe manner. Both results are reviewed with respect to the coming IEEE standard for interval arithmetic.

  @article{nehmeierSCAN2008, abstract = {In this paper we investigate how a C++ class library can be improved by the concept of expression templates. Our first result is a saving of rounding mode switches which considerably increases the performance. Our second result deals with handling the discontinuity flag that will probably be decided to be raised whenever a function is called outside its domain (loose evaluation). We discuss several alternatives and propose an expression related flag that can be used in a thread safe manner. Both results are reviewed with respect to the coming IEEE standard for interval arithmetic.}, author = {Nehmeier, Marco and Wolff von Gudenberg, J{\"u}rgen}, journal = {Reliable Computing}, keywords = {Reliability}, number = 4, pages = {312–320}, title = {filib++, {E}xpression {T}emplates and the {C}oming {I}nterval {S}tandard}, volume = 15, year = 2011 }
• 
  Automatyczna kompozycja uslug monitorowania dla systemu wspomagania treningu wytrzymalo’s}ciowego sportowców. K. Brzostowski; P. Rygielski; in Przeglka}d Telekomunikacyjny, Wiadomo’s}}ci Telekomunikacyjne (2011). 84 1017–1022.

  In Polish
  • [ BibTeX ]
  • [ Download ]
  @article{PT2011-kompozycja, author = {Brzostowski, Krzysztof and Rygielski, Piotr}, journal = {Przegl\k{a}d Telekomunikacyjny, Wiadomo{\'s}ci Telekomunikacyjne}, keywords = {t_journalmagazine}, note = {In Polish}, pages = {1017–1022}, title = {Automatyczna kompozycja uslug monitorowania dla systemu wspomagania treningu wytrzymalo\'sciowego sportowc\'ow}, volume = 84, year = 2011 }
• 
  Automated Transformation of Component-based Software Architecture Models to Queueing Petri Nets. P. Meier; S. Kounev; H. Koziolek; in 19th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2011), Singapore, July 25--27 (2011).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{MeKoKo2011-MASCOTS-PCMtoQPN, author = {Meier, Philipp and Kounev, Samuel and Koziolek, Heiko}, booktitle = {19th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2011), Singapore, July 25–27}, keywords = {QPN}, title = {Automated Transformation of Component-based Software Architecture Models to Queueing Petri Nets}, year = 2011 }
• 
  Engineering of Self-Aware IT Systems and Services: State-of-the-Art and Research Challenges. S. Kounev; in Proceedings of the 8th European Performance Engineering Workshop (EPEW’11), Borrowdale, The English Lake District, October 12--13 (2011).

  (Extended Abstract of Keynote Talk)
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{Ko2011-EPEW-ResearchChallenges, author = {Kounev, Samuel}, booktitle = {Proceedings of the 8th European Performance Engineering Workshop (EPEW'11), Borrowdale, The English Lake District, October 12–13}, keywords = {Reliability}, note = {(Extended Abstract of Keynote Talk)}, title = {{Engineering of Self-Aware IT Systems and Services: State-of-the-Art and Research Challenges}}, year = 2011 }
• 
  Defining and Quantifying Elasticity of Resources in Cloud Computing and Scalable Platforms M. Kuperberg; N. R. Herbst; J. G. von Kistowski; R. Reussner; (2011). (Vol. 16)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Elasticity is the ability of a software system to dynamically scale the amount of the resources it provides to clients as their workloads increase or decrease. Elasticity is praised as a key advantage of cloud computing, where computing resources are dynamically added and released. However, there exists no concise or formal definition of elasticity, and thus no approaches to quantify it have been developed so far. Existing work on cloud computing is limited to the technical view of implementing elastic systems, and definitions or scalability have not been extended to cover elasticity. In this report, we present a detailed discussion of elasticity, propose techniques for quantifying and measuring it, and outline next steps to be taken for enabling comparisons between cloud computing offerings on the basis of elasticity. We also present preliminary work on measuring elasticity of resource pools provided by the Java Virtual Machine.

  @techreport{KuHeKiRe2011-ResourceElasticity, abstract = {{Elasticity is the ability of a software system to dynamically scale the amount of the resources it provides to clients as their workloads increase or decrease. Elasticity is praised as a key advantage of cloud computing, where computing resources are dynamically added and released. However, there exists no concise or formal definition of elasticity, and thus no approaches to quantify it have been developed so far. Existing work on cloud computing is limited to the technical view of implementing elastic systems, and definitions or scalability have not been extended to cover elasticity. In this report, we present a detailed discussion of elasticity, propose techniques for quantifying and measuring it, and outline next steps to be taken for enabling comparisons between cloud computing offerings on the basis of elasticity. We also present preliminary work on measuring elasticity of resource pools provided by the Java Virtual Machine.}}, address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany}, author = {Kuperberg, Michael and Herbst, Nikolas Roman and von Kistowski, J{\'o}akim Gunnarsson and Reussner, Ralf}, institution = {Karlsruhe Institute of Technology (KIT)}, keywords = {t_techreport}, series = {Karlsruhe Reports in Informatics}, title = {{Defining and Quantifying Elasticity of Resources in Cloud Computing and Scalable Platforms}}, volume = 16, year = 2011 }
• 
  QPME 2.0 User’s Guide S. Kounev; S. Spinner; J. Walter; (2011). Karlsruhe Institute of Technology, Am Fasanengarten 5, 76131 Karlsruhe, Germany.
  • [ BibTeX ]
  @manual{KoSp2011-QPME20-UserGuide, address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany}, author = {Kounev, Samuel and Spinner, Simon and Walter, J{\"u}rgen}, keywords = {QPN}, month = {05}, organization = {Karlsruhe Institute of Technology}, title = {QPME 2.0 User's Guide}, year = 2011 }
• 
  Model-based Self-Adaptive Resource Allocation in Virtualized Environments. N. Huber; F. Brosig; S. Kounev; in 6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2011) (2011). 90–99.

  Acceptance Rate (Full Paper): 27% (21/76)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The adoption of virtualization and Cloud Computing technologies promises a number of benefits such as increased flexibility, better energy efficiency and lower operating costs for IT systems. However, highly variable workloads make it challenging to provide quality-of-service guarantees while at the same time ensuring efficient resource utilization. To avoid violations of service-level agreements (SLAs) or inefficient resource usage, resource allocations have to be adapted continuously during operation to reflect changes in application workloads. In this paper, we present a novel approach to self-adaptive resource allocation in virtualized environments based on online architecture-level performance models. We present a detailed case study of a representative enterprise application, the new SPECjEnterprise2010 benchmark, deployed in a virtualized cluster environment. The case study serves as a proof-of-concept demonstrating the effectiveness and practical applicability of our approach.

  @inproceedings{HuBrKo2011-SEAMS-ResAlloc, abstract = {The adoption of virtualization and Cloud Computing technologies promises a number of benefits such as increased flexibility, better energy efficiency and lower operating costs for IT systems. However, highly variable workloads make it challenging to provide quality-of-service guarantees while at the same time ensuring efficient resource utilization. To avoid violations of service-level agreements (SLAs) or inefficient resource usage, resource allocations have to be adapted continuously during operation to reflect changes in application workloads. In this paper, we present a novel approach to self-adaptive resource allocation in virtualized environments based on online architecture-level performance models. We present a detailed case study of a representative enterprise application, the new SPECjEnterprise2010 benchmark, deployed in a virtualized cluster environment. The case study serves as a proof-of-concept demonstrating the effectiveness and practical applicability of our approach.}, address = {New York, NY, USA}, author = {Huber, Nikolaus and Brosig, Fabian and Kounev, Samuel}, booktitle = {6th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS 2011)}, keywords = {Self-adaptive-systems}, month = {05}, note = {Acceptance Rate (Full Paper): 27\% (21/76)}, pages = {90–99}, publisher = {ACM}, title = {{Model-based Self-Adaptive Resource Allocation in Virtualized Environments}}, year = 2011 }
• 
  Ginpex: Deriving Performance-relevant Infrastructure Properties Through Goal-oriented Experiments. M. Hauck; M. Kuperberg; N. Huber; R. Reussner; in Proceedings of the 7th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2011) (2011). 53–62.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{hauck2011b, address = {New York, NY, USA}, author = {Hauck, Michael and Kuperberg, Michael and Huber, Nikolaus and Reussner, Ralf}, booktitle = {Proceedings of the 7th ACM SIGSOFT International Conference on the Quality of Software Architectures (QoSA 2011)}, keywords = {Virtualization}, month = {06}, pages = {53–62}, publisher = {ACM}, title = {{Ginpex: Deriving Performance-relevant Infrastructure Properties Through Goal-oriented Experiments}}, year = 2011 }
• 
  Palladio Workbench: A Quality-Prediction Tool for Component-Based Architectures. C. Rathfelder; B. Klatt; in Proceedings of the 2011 Ninth Working IEEE/IFIP Conference on Software Architecture (WICSA 2011) (2011). 347–350.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Today, software engineering is challenged to handle more and more large-scale distributed systems with a guaranteed level of service quality. Component-based architectures have been established to build more structured and manageable software systems. However, due to time and cost constraints, it is not feasible to use a trial and error approach to ensure that an architecture meets the quality of service (QoS) requirements. In this tool demo, we present the Palladio Workbench that permits the modeling of component-based software architectures and the prediction of its quality characteristics (e.g., response time and utilization). Additional to a general tool overview, we will give some insights about a new feature to analyze the impact of event-driven communication that was added in the latest release of the Palladio Component Model (PCM)

  @inproceedings{rathfelder2011a, abstract = {Today, software engineering is challenged to handle more and more large-scale distributed systems with a guaranteed level of service quality. Component-based architectures have been established to build more structured and manageable software systems. However, due to time and cost constraints, it is not feasible to use a trial and error approach to ensure that an architecture meets the quality of service (QoS) requirements. In this tool demo, we present the Palladio Workbench that permits the modeling of component-based software architectures and the prediction of its quality characteristics (e.g., response time and utilization). Additional to a general tool overview, we will give some insights about a new feature to analyze the impact of event-driven communication that was added in the latest release of the Palladio Component Model (PCM)}, address = {Washington, DC, USA}, author = {Rathfelder, Christoph and Klatt, Benjamin}, booktitle = {Proceedings of the 2011 Ninth Working IEEE/IFIP Conference on Software Architecture (WICSA 2011)}, keywords = {se2}, month = {06}, pages = {347–350}, publisher = {IEEE Computer Society}, title = {Palladio Workbench: A Quality-Prediction Tool for Component-Based Architectures}, year = 2011 }
• 
  Capacity Planning for Event-based Systems using Automated Performance Predictions. C. Rathfelder; S. Kounev; D. Evans; in 26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011) (2011). 352–361.

  Acceptance Rate (Full Paper): 14.7% (37/252)
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. The loose coupling of components in such systems makes it easy to vary the deployment. At the same time, the complexity to estimate the behavior and performance of the whole system is increased, which complicates capacity planning. In this paper, we present an automated performance prediction method supporting capacity planning for event-based systems. The performance prediction is based on an extended version of the Palladio Component Model -- a performance meta-model for component-based systems. We apply this method on a real-world case study of a traffic monitoring system. In addition to the application of our performance prediction techniques for capacity planning, we evaluate the prediction results against measurements in the context of the case study. The results demonstrate the practicality and effectiveness of the proposed approach.

  @inproceedings{RaKoEv2011-ASE-CapacityPlanning, abstract = {Event-based communication is used in different domains including telecommunications, transportation, and business information systems to build scalable distributed systems. The loose coupling of components in such systems makes it easy to vary the deployment. At the same time, the complexity to estimate the behavior and performance of the whole system is increased, which complicates capacity planning. In this paper, we present an automated performance prediction method supporting capacity planning for event-based systems. The performance prediction is based on an extended version of the Palladio Component Model – a performance meta-model for component-based systems. We apply this method on a real-world case study of a traffic monitoring system. In addition to the application of our performance prediction techniques for capacity planning, we evaluate the prediction results against measurements in the context of the case study. The results demonstrate the practicality and effectiveness of the proposed approach.}, author = {Rathfelder, Christoph and Kounev, Samuel and Evans, David}, booktitle = {26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011)}, keywords = {Application_quality_of_service_management}, month = 11, note = {Acceptance Rate (Full Paper): 14.7\% (37/252)}, pages = {352–361}, publisher = {IEEE}, title = {{C}apacity {P}lanning for {E}vent-based {S}ystems using {A}utomated {P}erformance {P}redictions}, year = 2011 }
• 
  Generative Programming for Automatic Differentiation in C++0x. (483), M. Nehmeier; (2011).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The computation of the first or higher derivatives of a function is a common problem in scientific computing. The most obvious approach to compute the derivative values is the symbolic differentiation which applies the well known rules of differentiation onto the expression to compute a formal expression of the derivative function. Another method called numerical differentiation is the approximation of the derivative with difference quotients. Superior to these approaches is the automatic differentiation which is a technique to compute a value of the expression and the derivative together by applying the well known rules of differentiation. The difference to symbolic differentiation is that it propagates numerical values instead of formal expressions. Commonly used implementations of automatic differentiation may be divided into two categories, the implementations using operator overloading to compute the values of the function and the derivative together in contrast to special tools applying the technique of source transformation to mix in the expressions to compute the derivatives. Both of these techniques are somehow inflexible in the manner that the operators are not overloaded for high order differentiation or additional tools are required for the source transformation. In this paper we use expression templates and template meta programming to mix in the code for the partial differentiation. The main concept is to apply the symbolic differentiation at compile time only onto the single operations of the expression tree to create the run time code for the automatic differentiation. This has two advantages, we can, in theory, compute derivatives of any order and secondly the differentiation of the operations is done with types. This means that the symbolic differentiation of an operation with a specified order is performed only once by the compiler. Our generic approach provides a domain specific language for partial automatic differentiation of an arbitrary order which is easily extendable by new types. Several C++ and especially C++0x template programming concepts and techniques like variadic templates, type lists or variadic tuples are used to realize the specification of the automatic differentiation at compile time. Common template meta programming techniques are refined. We tested the functionality, correctness and performance of our implementation in different case studies for floating point as well as interval data types and compared it against other implementations.

  @techreport{nehmeierTR483, abstract = {The computation of the first or higher derivatives of a function is a common problem in scientific computing. The most obvious approach to compute the derivative values is the symbolic differentiation which applies the well known rules of differentiation onto the expression to compute a formal expression of the derivative function. Another method called numerical differentiation is the approximation of the derivative with difference quotients. Superior to these approaches is the automatic differentiation which is a technique to compute a value of the expression and the derivative together by applying the well known rules of differentiation. The difference to symbolic differentiation is that it propagates numerical values instead of formal expressions. Commonly used implementations of automatic differentiation may be divided into two categories, the implementations using operator overloading to compute the values of the function and the derivative together in contrast to special tools applying the technique of source transformation to mix in the expressions to compute the derivatives. Both of these techniques are somehow inflexible in the manner that the operators are not overloaded for high order differentiation or additional tools are required for the source transformation. In this paper we use expression templates and template meta programming to mix in the code for the partial differentiation. The main concept is to apply the symbolic differentiation at compile time only onto the single operations of the expression tree to create the run time code for the automatic differentiation. This has two advantages, we can, in theory, compute derivatives of any order and secondly the differentiation of the operations is done with types. This means that the symbolic differentiation of an operation with a specified order is performed only once by the compiler. Our generic approach provides a domain specific language for partial automatic differentiation of an arbitrary order which is easily extendable by new types. Several C++ and especially C++0x template programming concepts and techniques like variadic templates, type lists or variadic tuples are used to realize the specification of the automatic differentiation at compile time. Common template meta programming techniques are refined. We tested the functionality, correctness and performance of our implementation in different case studies for floating point as well as interval data types and compared it against other implementations.}, author = {Nehmeier, Marco}, institution = {Department of Computer Science, University of W{\"u}rzburg}, keywords = {Reliability}, month = 12, number = 483, title = {{G}enerative {P}rogramming for {A}utomatic {D}ifferentiation in {C++0x}}, year = 2011 }
• 
  Performance Modeling for Quality of Service Prediction in Service-Oriented Systems C. Rathfelder; B. Klatt; F. Brosch; S. Kounev; in Handbook of Research on Service-Oriented Systems and Non-Functional Properties: Future Directions, (S. Reiff-Marganiec; M. Tilly, eds.) (2011). 258–279. IGI Global, Hershey, PA, USA.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  With the introduction of services, systems become more flexible as new services can easily be composed out of existing services. Services are increasingly used in mission-critical systems and applications and therefore considering Quality of Service (QoS) properties is an essential part of the service selection. Quality prediction techniques support the service provider in determining possible QoS levels that can be guaranteed to a customer or in deriving the operation costs induced by a certain QoS level. In this chapter, we present an overview on our work on modeling service-oriented systems for performance prediction using the Palladio Component Model. The prediction builds upon a model of a service-based system, and evaluates this model in order to determine the expected service quality. The presented techniques allow for early quality prediction, without the need for the system being already deployed and operating. We present the integration of our prediction approach into an SLA management framework. The emerging trend to combine event-based communication and Service-Oriented Architecture (SOA) into Event-based SOA (ESOA) induces new challenges to our approach, which are topic of a special subsection.

  @proceedings{rathfelder2010c, abstract = {With the introduction of services, systems become more flexible as new services can easily be composed out of existing services. Services are increasingly used in mission-critical systems and applications and therefore considering Quality of Service (QoS) properties is an essential part of the service selection. Quality prediction techniques support the service provider in determining possible QoS levels that can be guaranteed to a customer or in deriving the operation costs induced by a certain QoS level. In this chapter, we present an overview on our work on modeling service-oriented systems for performance prediction using the Palladio Component Model. The prediction builds upon a model of a service-based system, and evaluates this model in order to determine the expected service quality. The presented techniques allow for early quality prediction, without the need for the system being already deployed and operating. We present the integration of our prediction approach into an SLA management framework. The emerging trend to combine event-based communication and Service-Oriented Architecture (SOA) into Event-based SOA (ESOA) induces new challenges to our approach, which are topic of a special subsection.}, address = {Hershey, PA, USA}, author = {Rathfelder, Christoph and Klatt, Benjamin and Brosch, Franz and Kounev, Samuel}, booktitle = {{Handbook of Research on Service-Oriented Systems and Non-Functional Properties: Future Directions}}, editor = {Reiff-Marganiec, Stephan and Tilly, Marcel}, keywords = {t_bookchapter}, month = 12, pages = {258–279}, publisher = {IGI Global}, title = {{P}erformance {M}odeling for {Q}uality of {S}ervice {P}rediction in {S}ervice-{O}riented {S}ystems}, year = 2011 }
• 
  Quality Prediction in Service Composition Frameworks. B. Klatt; F. Brosch; Z. Durdik; C. Rathfelder; in 5th Workshop on Non-Functional Properties and SLA Management in Service-Oriented Computing (NFPSLAM-SOC 2011) (2011).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  With the introduction of services, software systems have become more flexible as new services can easily be composed from existing ones. Service composition frameworks offer corresponding functionality and hide the complexity of the underlying technologies from their users. However, possibilities for anticipating quality properties of com- posed services before their actual operation are limited so far. While existing approaches for model-based software quality prediction can be used by service composers for determining realizable Quality of Service (QoS) levels, integration of such techniques into composition frameworks is still missing. As a result, high effort and expert knowledge is required to build the system models required for prediction. In this paper, we present a novel service composition process that includes QoS prediction for composed services as an integral part. Furthermore, we describe how composition frameworks can be extended to support this process. With our approach, systematic consideration of service quality during the composition process is naturally achieved, without the need for de- tailed knowledge about the underlying prediction models. To evaluate our work and validate its applicability in different domains, we have integrated QoS prediction support according to our process in two com- position frameworks -- a large-scale SLA management framework and a service mashup platform.

  @inproceedings{klatt2011a, abstract = {With the introduction of services, software systems have become more flexible as new services can easily be composed from existing ones. Service composition frameworks offer corresponding functionality and hide the complexity of the underlying technologies from their users. However, possibilities for anticipating quality properties of com- posed services before their actual operation are limited so far. While existing approaches for model-based software quality prediction can be used by service composers for determining realizable Quality of Service (QoS) levels, integration of such techniques into composition frameworks is still missing. As a result, high effort and expert knowledge is required to build the system models required for prediction. In this paper, we present a novel service composition process that includes QoS prediction for composed services as an integral part. Furthermore, we describe how composition frameworks can be extended to support this process. With our approach, systematic consideration of service quality during the composition process is naturally achieved, without the need for de- tailed knowledge about the underlying prediction models. To evaluate our work and validate its applicability in different domains, we have integrated QoS prediction support according to our process in two com- position frameworks – a large-scale SLA management framework and a service mashup platform.}, author = {Klatt, Benjamin and Brosch, Franz and Durdik, Zoya and Rathfelder, Christoph}, booktitle = {5th Workshop on Non-Functional Properties and SLA Management in Service-Oriented Computing (NFPSLAM-SOC 2011)}, keywords = {t_workshop}, month = 12, title = {{Quality Prediction in Service Composition Frameworks}}, year = 2011 }
• 
  Resources Management and Services Personalization in Future Internet Applications. P. ’S}}wika}tek; P. Rygielski; A. Grzech; in Advances in Electronics and Telecommunications (2011). 2(3) 50–54.
  • [ BibTeX ]
  • [ URL ]
  @article{SwRyGr2011-EAT, address = {Pozna{\'n}, Poland}, author = {{\'S}wi\k{a}tek, Pawel and Rygielski, Piotr and Grzech, Adam}, journal = {Advances in Electronics and Telecommunications}, keywords = {t_journalmagazine}, month = 11, number = 3, pages = {50–54}, publisher = {Pozna{\'n} University of Technology Press}, title = {{Resources Management and Services Personalization in Future Internet Applications}}, volume = 2, year = 2011 }
• 
  Model-based Self-Adaptive Resource Management in Virtualized Environments. S. Kounev; (2011, June).
  • [ BibTeX ]
  @misc{talk-Kounev-VMWare11, author = {Kounev, Samuel}, howpublished = {Invited Talk, VMware, Inc., USA}, keywords = {Talks}, month = {06}, title = {{Model-based Self-Adaptive Resource Management in Virtualized Environments}}, year = 2011 }
• 
  Universal Comunication Platform for QoS-aware Delivery of Complex Services. P. ’S}}wika}tek; P. Rygielski; in Proceedings of the VIth International Scientific and Technical Conference (CSIT 2011) (2011). 136–139.
  • [ BibTeX ]
  @inproceedings{SwRy2011-CIST, author = {{\'S}wi\k{a}tek, Pawe{\l} and Rygielski, Piotr}, booktitle = {{Proceedings of the VIth International Scientific and Technical Conference (CSIT 2011)}}, keywords = {Virtualization}, month = 11, pages = {136–139}, publisher = {Publishing House Vezha&Co}, title = {{Universal Comunication Platform for QoS-aware Delivery of Complex Services}}, year = 2011 }
• 
  Descartes Research Group at KIT: Engineering of Self-Aware IT Systems and Services. S. Kounev; (2011, November).
  • [ BibTeX ]
  @misc{talk-Kounev-BYonder11, author = {Kounev, Samuel}, howpublished = {Invited Talk, Blue Yonder GmbH & Co. KG, Karlsruhe, Germany}, keywords = {Talks}, month = 11, title = {{Descartes Research Group at KIT: Engineering of Self-Aware IT Systems and Services}}, year = 2011 }
• 
  From Offline to Online Component Models for Run-Time Resource Management. S. Kounev; (2011, November).
  • [ BibTeX ]
  @misc{talk-Kounev-Palladio11, author = {Kounev, Samuel}, howpublished = {Invited Talk, Palladio Days 2011, FZI Forschungzentrum Informatik, Karlsruhe, Germany}, keywords = {Talks}, month = 11, title = {{From Offline to Online Component Models for Run-Time Resource Management}}, year = 2011 }
• 
  Automated Extraction of Architecture-Level Performance Models of Distributed Component-Based Systems. F. Brosig; N. Huber; S. Kounev; in 26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011) (2011).

  Acceptance Rate (Full Paper): 14.7% (37/252)
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Modern service-oriented enterprise systems have increasingly complex and dynamic loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment and adapt the system configuration accordingly. Architecture-level performance models provide a powerful tool for performance prediction, however, current approaches to modeling the execution context of software components are not suitable for use at run-time. In this paper, we analyze the typical online performance prediction scenarios and propose a novel performance meta-model for expressing and resolving parameter and context dependencies, specifically designed for use in online scenarios. We motivate and validate our approach in the context of a realistic and representative online performance prediction scenario based on the SPECjEnterprise2010 standard benchmark.

  @inproceedings{BrHuKo2011-ASE-AutomExtraction, abstract = {Modern service-oriented enterprise systems have increasingly complex and dynamic loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment and adapt the system configuration accordingly. Architecture-level performance models provide a powerful tool for performance prediction, however, current approaches to modeling the execution context of software components are not suitable for use at run-time. In this paper, we analyze the typical online performance prediction scenarios and propose a novel performance meta-model for expressing and resolving parameter and context dependencies, specifically designed for use in online scenarios. We motivate and validate our approach in the context of a realistic and representative online performance prediction scenario based on the SPECjEnterprise2010 standard benchmark.}, address = {Oread, Lawrence, Kansas}, author = {Brosig, Fabian and Huber, Nikolaus and Kounev, Samuel}, booktitle = {26th IEEE/ACM International Conference On Automated Software Engineering (ASE 2011)}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = 11, note = {Acceptance Rate (Full Paper): 14.7\% (37/252)}, title = {{A}utomated {E}xtraction of {A}rchitecture-{L}evel {P}erformance {M}odels of {D}istributed {C}omponent-{B}ased {S}ystems}, year = 2011 }
• 
  Engineering of Self-Aware IT Systems and Services. S. Kounev; (2011, November).
  • [ BibTeX ]
  @misc{talk-Kounev-InvImperial11, author = {Kounev, Samuel}, howpublished = {Invited Talk, Imperial College London, Department of Computing, London, UK}, keywords = {Talks}, month = 11, title = {{Engineering of Self-Aware IT Systems and Services}}, year = 2011 }
• 
  Engineering of Self-Aware IT Systems and Services: State-of-the-Art and Research Challenges. S. Kounev; (2011, October).
  • [ BibTeX ]
  @misc{talk-Kounev-EPEW11, author = {Kounev, Samuel}, howpublished = {Keynote Talk, 8th European Performance Engineering Workshop, Borrowdale, The English Lake District}, keywords = {Talks}, month = 10, title = {{Engineering of Self-Aware IT Systems and Services: State-of-the-Art and Research Challenges}}, year = 2011 }
• 
  Context Change Detection for Resource Allocation in Service-Oriented Systems. P. Rygielski; J. Tomczak; in Knowledge-Based and Intelligent Information and Engineering Systems, Part II (KES 2011), A. Koenig, A. Dengel, K. Hinkelmann, K. Kise, R. Howlett, L. Jain (eds.) (2011). (1st ed., Vol. 6882) 591–600.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper, the problem of detecting the major changes in the stream of service requests is formulated. The change of stream component varies over time and depends on, e.g., a time of a day. The underlying cause of the change is called a context. Hence, at each moment there exists a probability distribution determining the probability of requesting the system service conditioned by the context. The aim is to find such a moment in which the distributions change. To solve that problem dissimilarity measures between two probability distributions are given. Nevertheless, detecting every change is not interesting but only long-lasting changes because of the costs of the service system resources reallocation. Therefore, in the proposed algorithm an issue of sensitivity to temporary changes detection is considered

  @inproceedings{RyTo2011-KES, abstract = {In this paper, the problem of detecting the major changes in the stream of service requests is formulated. The change of stream component varies over time and depends on, e.g., a time of a day. The underlying cause of the change is called a context. Hence, at each moment there exists a probability distribution determining the probability of requesting the system service conditioned by the context. The aim is to find such a moment in which the distributions change. To solve that problem dissimilarity measures between two probability distributions are given. Nevertheless, detecting every change is not interesting but only long-lasting changes because of the costs of the service system resources reallocation. Therefore, in the proposed algorithm an issue of sensitivity to temporary changes detection is considered}, address = {Berlin, Heidelberg}, author = {Rygielski, Piotr and Tomczak, Jakub}, booktitle = {Knowledge-Based and Intelligent Information and Engineering Systems, Part II (KES 2011)}, edition = 1, editor = {Koenig, A. and Dengel, A. and Hinkelmann, K. and Kise, K. and Howlett, R.J. and Jain, L.C.}, keywords = {descartes}, month = {09}, pages = {591–600}, publisher = {Springer-Verlag}, series = {{Lecture Notes in Computer Science (LNCS)}}, title = {{Context Change Detection for Resource Allocation in Service-Oriented Systems}}, volume = 6882, year = 2011 }
• 
  Fifth International Workshop on the Practical Application of Stochastic Modelling (PASM) F. Brosig; in Electronic Notes in Theoretical Computer Science, (N. Thomas; J. Bradley; W. Knottenbelt; S. Kounev; N. Huber; F. Brosig, eds.) (2011). (Vol. 275) 1–174. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
  • [ BibTeX ]
  @proceedings{Thomas20111, address = {Amsterdam, The Netherlands}, author = {Brosig, Fabian}, editor = {Thomas, Nigel and Bradley, Jeremy and Knottenbelt, William and Kounev, Samuel and Huber, Nikolaus and Brosig, Fabian}, keywords = {Reliability}, month = {09}, pages = {1–174}, publisher = {Elsevier Science Publishers B. V.}, series = {Electronic Notes in Theoretical Computer Science}, title = {Fifth International Workshop on the Practical Application of Stochastic Modelling (PASM)}, volume = 275, year = 2011 }
• 
  Prediction Based Handovers for Wireless Networks Resources Management. P. Rygielski; P. ’S}}wika}tek; K. Juszczyszyn; A. Grzech; in Knowledge-Based and Intelligent Information and Engineering Systems, Part II (KES 2011), A. Koenig, A. Dengel, K. Hinkelmann, K. Kise, R. Howlett, L. Jain (eds.) (2011). (1st ed., Vol. 6882) 687–696.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The paper is devoted to the problem of mobility and resources management in heterogeneous wireless networks. It is assumed that in certain area covered by multiple overlapping wireless networks there is certain number of mobile client which consume networks resources by use of available communication services (e.g., voice or data transmission) delivered by network providers. Moreover it is assumed, that communication services continuity may be assured with use of common handover techniques supporting clients mobility (e.g: MIPv6, IEEE 802.21, etc.). The task of mobility and resources management consists of making decisions concerning the moment and the network to which particular clients should be handed over in order to optimize certain quality criterion (e.g. utilization of network resources). In this paper we show that gathering knowledge about clients movement and prediction of their future position may significantly improve the overall quality of delivered services and network resources utilization.

  @inproceedings{RySwJuGr2011-KES, abstract = {The paper is devoted to the problem of mobility and resources management in heterogeneous wireless networks. It is assumed that in certain area covered by multiple overlapping wireless networks there is certain number of mobile client which consume networks resources by use of available communication services (e.g., voice or data transmission) delivered by network providers. Moreover it is assumed, that communication services continuity may be assured with use of common handover techniques supporting clients mobility (e.g: MIPv6, IEEE 802.21, etc.). The task of mobility and resources management consists of making decisions concerning the moment and the network to which particular clients should be handed over in order to optimize certain quality criterion (e.g. utilization of network resources). In this paper we show that gathering knowledge about clients movement and prediction of their future position may significantly improve the overall quality of delivered services and network resources utilization.}, address = {Berlin, Heidelberg}, author = {Rygielski, Piotr and {\'S}wi\k{a}tek, Pawel and Juszczyszyn, Krzysztof and Grzech, Adam}, booktitle = {Knowledge-Based and Intelligent Information and Engineering Systems, Part II (KES 2011)}, edition = 1, editor = {Koenig, A. and Dengel, A. and Hinkelmann, K. and Kise, K. and Howlett, R.J. and Jain, L.C.}, keywords = {Prediction}, month = {09}, pages = {687–696}, publisher = {Springer-Verlag}, series = {{Lecture Notes in Computer Science (LNCS)}}, title = {{Prediction Based Handovers for Wireless Networks Resources Management}}, volume = 6882, year = 2011 }
• 
  Fifth International Workshop on the Practical Application of Stochastic Modelling (PASM) N. Huber; in Electronic Notes in Theoretical Computer Science, (N. Thomas; J. Bradley; W. Knottenbelt; S. Kounev; N. Huber; F. Brosig, eds.) (2011). (Vol. 275) 1–174. Elsevier Science Publishers B. V., Amsterdam, The Netherlands.
  • [ BibTeX ]
  @proceedings{Thomas20111, address = {Amsterdam, The Netherlands}, author = {Huber, Nikolaus}, editor = {Thomas, Nigel and Bradley, Jeremy and Knottenbelt, William and Kounev, Samuel and Huber, Nikolaus and Brosig, Fabian}, keywords = {Reliability}, month = {09}, pages = {1–174}, publisher = {Elsevier Science Publishers B. V.}, series = {Electronic Notes in Theoretical Computer Science}, title = {Fifth International Workshop on the Practical Application of Stochastic Modelling (PASM)}, volume = 275, year = 2011 }
• 
  Migration-aware Optimization of Virtualized Computational Resources Allocation in Complex Systems. P. Rygielski; A. Gonczarek; in ICSEng 2011: Proceedings of the 21st international conference on Systems Engineering (ICSEng 2011), H. Selvaraj, D. Zydek (eds.) (2011). 212–216.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper a task of resources allocation in the complex system is considered. Novelty of the formulated task consists of assumption that the applications assigned with resources of one machine can be migrated to another machine during system lifetime. The formulated task has been solved using proposed heuristic optimization method. Due to non-convex set of valid solutions the optimization procedure has been decomposed into two stages and forms approach similar to the relax-and-round approach. Proposed decomposition approach facilitates fast algorithm convergence and guaranties that achieved solution satisfies assumed constraints

  @inproceedings{RyGo2011-ICSEng, abstract = {In this paper a task of resources allocation in the complex system is considered. Novelty of the formulated task consists of assumption that the applications assigned with resources of one machine can be migrated to another machine during system lifetime. The formulated task has been solved using proposed heuristic optimization method. Due to non-convex set of valid solutions the optimization procedure has been decomposed into two stages and forms approach similar to the relax-and-round approach. Proposed decomposition approach facilitates fast algorithm convergence and guaranties that achieved solution satisfies assumed constraints}, address = {Los Alamitos, California USA}, author = {Rygielski, Piotr and Gonczarek, Adam}, booktitle = {ICSEng 2011: Proceedings of the 21st international conference on Systems Engineering (ICSEng 2011)}, editor = {Selvaraj, Henry and Zydek, Dawid}, keywords = {Optimization}, month = {08}, pages = {212–216}, publisher = {IEEE Computer Society's Conference Publishing Services}, title = {{Migration-aware Optimization of Virtualized Computational Resources Allocation in Complex Systems}}, year = 2011 }
• 
  Performance Engineering of Business Information Systems - Filling the Gap between High-level Business Services and Low-level Performance Models. S. Kounev; in International Symposium on Business Modeling and Software Design (BMSD 2011), Sofia, Bulgaria, July 27--28, 2011 (2011).
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{Ko2011-BMSD-PerfEngOfBis, author = {Kounev, Samuel}, booktitle = {International Symposium on Business Modeling and Software Design (BMSD 2011), Sofia, Bulgaria, July 27–28, 2011}, keywords = {Application_quality_of_service_management}, month = {07}, title = {{Performance Engineering of Business Information Systems - Filling the Gap between High-level Business Services and Low-level Performance Models}}, year = 2011 }
• 
  Self-Aware QoS Management in Virtualized Infrastructures. S. Kounev; F. Brosig; N. Huber; in 8th International Conference on Autonomic Computing (ICAC 2011) (2011).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  We present an overview of our work-in-progress and long-term research agenda aiming to develop a novel methodology for engineering of self-aware software systems. The latter will have built-in architecture-level QoS models enhanced to capture dynamic aspects of the system environment and maintained automatically during operation. The models will be exploited at run-time to adapt the system to changes in the environment ensuring that resources are utilized efficiently and QoS requirements are satisfied.

  @inproceedings{KoBrHu2011-ICAC-QoSManagement, abstract = {We present an overview of our work-in-progress and long-term research agenda aiming to develop a novel methodology for engineering of self-aware software systems. The latter will have built-in architecture-level QoS models enhanced to capture dynamic aspects of the system environment and maintained automatically during operation. The models will be exploited at run-time to adapt the system to changes in the environment ensuring that resources are utilized efficiently and QoS requirements are satisfied.}, author = {Kounev, Samuel and Brosig, Fabian and Huber, Nikolaus}, booktitle = {8th International Conference on Autonomic Computing (ICAC 2011)}, keywords = {Reliability}, month = {06}, title = {{Self-Aware QoS Management in Virtualized Infrastructures}}, year = 2011 }
• 
  Efficient Experiment Selection in Automated Software Performance Evaluations. D. Westermann; R. Krebs; J. Happe; in Proceedings of the Computer Performance Engineering - 8th European Performance Engineering Workshop (EPEW 2011) (2011). 325–339.
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{Westermann2011b, author = {Westermann, Dennis and Krebs, Rouven and Happe, Jens}, booktitle = {Proceedings of the Computer Performance Engineering - 8th European Performance Engineering Workshop (EPEW 2011)}, keywords = {t_workshop}, month = 10, pages = {325-339}, publisher = {Springer}, title = {{E}fficient {E}xperiment {S}election in {A}utomated {S}oftware {P}erformance {E}valuations}, year = 2011 }
• 
  Integration of event-based communication in the palladio software quality prediction framework. B. Klatt; C. Rathfelder; S. Kounev; in Proceedings of the joint ACM SIGSOFT conference -- QoSA and ACM SIGSOFT symposium -- ISARCS on Quality of software architectures -- QoSA and architecting critical systems -- ISARCS (QoSA-ISARCS 2011) (2011). 43–52.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Today, software engineering is challenged to handle more and more large-scale distributed systems with guaranteed quality-of-service. Component-based architectures have been established to build such systems in a more structured and manageable way. Modern architectures often utilize event-based communication which enables loosely-coupled interactions between components and leads to improved system scalability. However, the loose coupling of components makes it challenging to model such architectures in order to predict their quality properties, e.g., performance and reliability, at system design time. In this paper, we present an extension of the Palladio Component Model (PCM) and the Palladio software quality prediction framework, enabling the modeling of event-based communication in component-based architectures. The contributions include: i) a meta-model extension supporting events as first class entities, ii) a model-to-model transformation from the extended to the original PCM, iii) an integration of the transformation into the Palladio tool chain allowing to use existing model solution techniques, and iv) a detailed evaluation of the reduction of the modeling effort enabled by the transformation in the context of a real-world case study.

  @inproceedings{KlRaKo2011-QoSA-PCMEvents, abstract = {Today, software engineering is challenged to handle more and more large-scale distributed systems with guaranteed quality-of-service. Component-based architectures have been established to build such systems in a more structured and manageable way. Modern architectures often utilize event-based communication which enables loosely-coupled interactions between components and leads to improved system scalability. However, the loose coupling of components makes it challenging to model such architectures in order to predict their quality properties, e.g., performance and reliability, at system design time. In this paper, we present an extension of the Palladio Component Model (PCM) and the Palladio software quality prediction framework, enabling the modeling of event-based communication in component-based architectures. The contributions include: i) a meta-model extension supporting events as first class entities, ii) a model-to-model transformation from the extended to the original PCM, iii) an integration of the transformation into the Palladio tool chain allowing to use existing model solution techniques, and iv) a detailed evaluation of the reduction of the modeling effort enabled by the transformation in the context of a real-world case study.}, address = {New York, NY, USA}, author = {Klatt, Benjamin and Rathfelder, Christoph and Kounev, Samuel}, booktitle = {Proceedings of the joint ACM SIGSOFT conference – QoSA and ACM SIGSOFT symposium – ISARCS on Quality of software architectures – QoSA and architecting critical systems – ISARCS (QoSA-ISARCS 2011)}, keywords = {Application_quality_of_service_management}, month = {06}, organization = {SIGSOFT}, pages = {43–52}, publisher = {ACM}, title = {Integration of event-based communication in the palladio software quality prediction framework}, year = 2011 }

• 
  QPME 2.0 - A Tool for Stochastic Modeling and Analysis Using Queueing Petri Nets. S. Kounev; S. Spinner; P. Meier; in From Active Data Management to Event-Based Systems and More, K. Sachs, I. Petrov, P. Guerrero (eds.) (2010). (Vol. 6462) 293–311.
  • [ BibTeX ]
  @incollection{KoSpMe2010-Festschrift-QPME2, address = {Berlin, Heidelberg}, author = {Kounev, Samuel and Spinner, Simon and Meier, Philipp}, booktitle = {From Active Data Management to Event-Based Systems and More}, editor = {Sachs, Kai and Petrov, Ilia and Guerrero, Pablo}, keywords = {QPN}, pages = {293–311}, publisher = {Springer-Verlag}, series = {Lecture Notes in Computer Science}, title = {{QPME 2.0 - A Tool for Stochastic Modeling and Analysis Using Queueing Petri Nets}}, volume = 6462, year = 2010 }
• 
  Benchmarking Publish/Subscribe-based Messaging Systems. K. Sachs; S. Appel; S. Kounev; A. Buchmann; in Proc. of 2nd International Workshop on Benchmarking of Database Management Systems and Data-Oriented Web Technologies (BenchmarX’10)., M. Necasky, E. Pardede (eds.) (2010). (Vol. 6193)
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{SaApKoBu2010-BenchmarX-PubSub, author = {Sachs, Kai and Appel, Stefan and Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proc. of 2nd International Workshop on Benchmarking of Database Management Systems and Data-Oriented Web Technologies (BenchmarX'10).}, editor = {Necasky, Martin and Pardede, Eric}, keywords = {t_workshop}, month = {04}, publisher = {Springer}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Benchmarking Publish/Subscribe-based Messaging Systems}}, volume = 6193, year = 2010 }
• 
  Comprehensive tool support for iterative SOA evolution. T. Schuster; C. Rathfelder; N. Schuster; J. Nimis; in Proceedings of the International Workshop on SOA Migration and Evolution 2010 (SOAME 2010) as part of the 14th European Conference on Software Maintenance and Reengineering (CSMR 2010) (2010). 1–10.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In recent years continuously changing market situations required IT systems that are flexible and highly responsive to changes of the underlying business processes. The transformation to service-oriented architecture (SOA) concepts, mainly services and loose coupling, promises to meet these demands. However, elevated complexity in management and evolution processes is required for the migration of existing systems towards SOA. Studies in this area of research have revealed a gap between in continuous and actual tool support of development teams throughout the process phases of evolution processes. Thus, in this article we introduce a method that fosters evolution by an iterative approach and illustrate how each phase of this method can be tool-supported.

  @inproceedings{schuster2010a, abstract = {In recent years continuously changing market situations required IT systems that are flexible and highly responsive to changes of the underlying business processes. The transformation to service-oriented architecture (SOA) concepts, mainly services and loose coupling, promises to meet these demands. However, elevated complexity in management and evolution processes is required for the migration of existing systems towards SOA. Studies in this area of research have revealed a gap between in continuous and actual tool support of development teams throughout the process phases of evolution processes. Thus, in this article we introduce a method that fosters evolution by an iterative approach and illustrate how each phase of this method can be tool-supported.}, author = {Schuster, Thomas and Rathfelder, Christoph and Schuster, Nelly and Nimis, Jens}, booktitle = {Proceedings of the International Workshop on SOA Migration and Evolution 2010 (SOAME 2010) as part of the 14th European Conference on Software Maintenance and Reengineering (CSMR 2010)}, keywords = {t_workshop}, month = {03}, pages = {1–10}, title = {Comprehensive tool support for iterative SOA evolution}, year = 2010 }
• 
  METHOD AND SYSTEM FOR FOR AN ADAPTIVE LEARNING STRATEGY. R. Krebs; (2010, March).
  • [ BibTeX ]
  @misc{Kr2010-uspatent-adaptive-elearning, author = {Krebs, Rouven}, howpublished = {Patent US8108327}, keywords = {se2}, month = {03}, title = {METHOD AND SYSTEM FOR FOR AN ADAPTIVE LEARNING STRATEGY}, year = 2010 }
• 
  QoS-aware Infrastructure Resources Allocation in Systems Based on Service-Oriented Architecture Paradigm. A. Grzech; P. Rygielski; P. ’S}}wika}tek; in Performance modelling and evaluation of heterogeneous networks, 6th Working International Conference (HET-NETs 2010), T. Czachórski (ed.) (2010). 35–47.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper the task of communication and computational resources allocation in systems based on SOA paradigm is considered. The task of resources allocation consists in assigning resources to each incoming service request in such a way, that required level of the quality of service is met. Complex services performance time in distributed environment is assumed as the quality of service measure. Models of the system and analysis of service response time presented in this paper allowed to formulate several tasks of resource allocation in terms of well known quality of service assurance models: best effort, IntServ and DiffServ. For each formulated task solution algorithms were proposed and their correctness was evaluated by means of simulation.

  @inproceedings{GrRySw2009-HETNETS-QoS, abstract = {In this paper the task of communication and computational resources allocation in systems based on SOA paradigm is considered. The task of resources allocation consists in assigning resources to each incoming service request in such a way, that required level of the quality of service is met. Complex services performance time in distributed environment is assumed as the quality of service measure. Models of the system and analysis of service response time presented in this paper allowed to formulate several tasks of resource allocation in terms of well known quality of service assurance models: best effort, IntServ and DiffServ. For each formulated task solution algorithms were proposed and their correctness was evaluated by means of simulation.}, address = {Ba{l}tycka 5, 44-100 Gliwice, Poland}, author = {Grzech, Adam and Rygielski, Piotr and {\'S}wi\k{a}tek, Pawel}, booktitle = {Performance modelling and evaluation of heterogeneous networks, 6th Working International Conference (HET-NETs 2010)}, editor = {Czach{\'o}rski, Tadeusz}, keywords = {descartes}, month = {01}, pages = {35–47}, publisher = {Institute of Theoretical and Applied Informatics of the Polish Academy of Sciences}, title = {{QoS-aware Infrastructure Resources Allocation in Systems Based on Service-Oriented Architecture Paradigm}}, year = 2010 }
• 
  Simulation Environment for Delivering Quality of Service in Systems Based on Service-Oriented Architecture Paradigm. A. Grzech; P. Rygielski; P. ’S}}wika}tek; in Performance modeling and evaluation of heterogeneous networks, 6th Working International Conference (HET-NETs 2010), T. Czachórski (ed.) (2010). 89–97.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper a model of complex service in service-oriented architecture (SOA) system is presented. A complex service is composed with a set of atomic services. Each atomic service is characterized with its own non-functional parameters what allows to formulate quality of service optimization tasks. A simulation environment has been developed to allow experiments execution to determine quality of service (QoS) of composed service.

  @inproceedings{GrRySw2009-HETNETS-sim, abstract = {In this paper a model of complex service in service-oriented architecture (SOA) system is presented. A complex service is composed with a set of atomic services. Each atomic service is characterized with its own non-functional parameters what allows to formulate quality of service optimization tasks. A simulation environment has been developed to allow experiments execution to determine quality of service (QoS) of composed service.}, address = {Ba{l}tycka 5, 44-100 Gliwice, Poland}, author = {Grzech, Adam and Rygielski, Piotr and {\'S}wi\k{a}tek, Pawel}, booktitle = {Performance modeling and evaluation of heterogeneous networks, 6th Working International Conference (HET-NETs 2010)}, editor = {Czach{\'o}rski, Tadeusz}, keywords = {descartes}, month = {01}, pages = {89–97}, publisher = {Institute of Theoretical and Applied Informatics of the Polish Academy of Sciences}, title = {{Simulation Environment for Delivering Quality of Service in Systems Based on Service-Oriented Architecture Paradigm}}, year = 2010 }
• 
  Performance Modeling and Benchmarking of Event-Based Systems. K. Sachs; Thesis; TU Darmstadt; Karolinenplatz 5, 64289 Darmstadt, Germany. (2010).
  • [ BibTeX ]
  • [ Download ]
  @phdthesis{KaiSachs-Thesis-2010, address = {Karolinenplatz 5, 64289 Darmstadt, Germany}, author = {Sachs, Kai}, keywords = {QPME_Bibliography}, school = {TU Darmstadt}, title = {Performance Modeling and Benchmarking of Event-Based Systems}, year = 2010 }
• 
  QoS-aware Complex Service Composition in SOA-based Systems. A. Grzech; P. Rygielski; P. ’S}}wika}tek; in SOA Infrastructure Tools: Concepts and Methods, S. Ambroszkiewicz, J. Brzeziński, W. Cellary, A. Grzech, K. Zieli"n}}ski (eds.) (2010). 289–312.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this work, a method for the QoS-aware complex service composition in SOA-based systems is presented. The considered complex service composition process consists of three stages: complex service structure, scenario and plan composition. Complex service structure contains a set of required functionalities and precedence relations between them, and is derived directly from a service level agreement. Service scenario is derived from service structure by choosing the optimal (in the sense of certain quality criterion) order of execution of required functionalities. Finally, a service plan emerges from a scenario by choosing the best versions of atomic services for the delivery of required functionalities. For such distinguished composition process methods for complex service scenario and execution plan optimization are proposed. Optimization of a complex service scenario is based on the analysis of possible parallel executions of required functionalities. In order to deliver required level of the quality of service, well known QoS assurance models (i.e.: best effort, integrated services and differentiated services) are applied in the process of complex service execution plan optimization.

  @incollection{GrRySw2010-SOAbook, abstract = {In this work, a method for the QoS-aware complex service composition in SOA-based systems is presented. The considered complex service composition process consists of three stages: complex service structure, scenario and plan composition. Complex service structure contains a set of required functionalities and precedence relations between them, and is derived directly from a service level agreement. Service scenario is derived from service structure by choosing the optimal (in the sense of certain quality criterion) order of execution of required functionalities. Finally, a service plan emerges from a scenario by choosing the best versions of atomic services for the delivery of required functionalities. For such distinguished composition process methods for complex service scenario and execution plan optimization are proposed. Optimization of a complex service scenario is based on the analysis of possible parallel executions of required functionalities. In order to deliver required level of the quality of service, well known QoS assurance models (i.e.: best effort, integrated services and differentiated services) are applied in the process of complex service execution plan optimization.}, address = {Pozna{\'n}, Poland}, author = {Grzech, Adam and Rygielski, Piotr and {\'S}wi\k{a}tek, Pawel}, booktitle = {SOA Infrastructure Tools: Concepts and Methods}, editor = {Ambroszkiewicz, Stanislaw and Brzezi{\'n}ski, Jerzy and Cellary, Wojciech and Grzech, Adam and Zieli{\"n}ski, Krzysztof}, keywords = {descartes}, pages = {289–312}, publisher = {Pozna{\'n} University of Economics Press}, title = {{QoS-aware Complex Service Composition in SOA-based Systems}}, year = 2010 }
• 
  Statistical Inference of Software Performance Models for Parametric Performance Completions. J. Happe; D. Westermann; K. Sachs; L. Kapova; in Research into Practice - Reality and Gaps (Proceedings of QoSA 2010), G. Heineman, J. Kofron, F. Plasil (eds.) (2010). (Vol. 6093) 20–35.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Software performance engineering (SPE) enables software architects to ensure high performance standards for their applications. However, applying SPE in practice is still challenging. Most enterprise applications include a large software basis, such as middleware and legacy systems. In many cases, the software basis is the determining factor of the system's overall timing behavior, throughput, and resource utilization. To capture these influences on the overall system's performance, established performance prediction methods (modelbased and analytical) rely on models that describe the performance-relevant aspects of the system under study. Creating such models requires detailed knowledge on the system's structure and behavior that, in most cases, is not available. In this paper, we abstract from the internal structure of the system under study. We focus our efforts on message-oriented middleware and analyze the dependency between the MOM's usage and its performance. We use statistical inference to conclude these dependencies from observations. For ActiveMQ 5.3, the resulting functions predict the performance with an relative mean square error 0.1.

  @inproceedings{happe2010a, abstract = {Software performance engineering (SPE) enables software architects to ensure high performance standards for their applications. However, applying SPE in practice is still challenging. Most enterprise applications include a large software basis, such as middleware and legacy systems. In many cases, the software basis is the determining factor of the system's overall timing behavior, throughput, and resource utilization. To capture these influences on the overall system's performance, established performance prediction methods (modelbased and analytical) rely on models that describe the performance-relevant aspects of the system under study. Creating such models requires detailed knowledge on the system's structure and behavior that, in most cases, is not available. In this paper, we abstract from the internal structure of the system under study. We focus our efforts on message-oriented middleware and analyze the dependency between the MOM's usage and its performance. We use statistical inference to conclude these dependencies from observations. For ActiveMQ 5.3, the resulting functions predict the performance with an relative mean square error 0.1.}, author = {Happe, Jens and Westermann, Dennis and Sachs, Kai and Kapova, Lucia}, booktitle = {{Research into Practice - Reality and Gaps (Proceedings of QoSA 2010)}}, editor = {Heineman, George and Kofron, Jan and Plasil, Frantisek}, keywords = {se2}, pages = {20–35}, publisher = {Springer}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Statistical Inference of Software Performance Models for Parametric Performance Completions}}, volume = 6093, year = 2010 }
• 
  Challenges and Opportunities of Cloud Computing -- Trade-off Decisions in Cloud Computing Architecture. (2010-19), M. Hauck; M. Huber; M. Klems; S. Kounev; J. Müller-Quade; A. Pretschner; R. Reussner; S. Tai; (2010).
  • [ BibTeX ]
  • [ URL ]
  @techreport{hauck2010b, author = {Hauck, Michael and Huber, Matthias and Klems, Markus and Kounev, Samuel and M{\"u}ller-Quade, J{\"o}rn and Pretschner, Alexander and Reussner, Ralf and Tai, Stefan}, institution = {Karlsruhe Institue of Technology, Faculty of Informatics}, keywords = {t_bookchapter}, number = {2010-19}, title = {{Challenges and Opportunities of Cloud Computing – Trade-off Decisions in Cloud Computing Architecture}}, year = 2010 }
• 
  Using Quality of Service Bounds for Effective Multi-objective Software Architecture Optimization. Q. Noorshams; A. Martens; R. Reussner; in Proceedings of the 2nd International Workshop on the Quality of Service-Oriented Software Systems (QUASOSS ’10), Oslo, Norway, October 4, 2010 (2010). 1:1–1:6.
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{NoMaRe2010-QUASOSS-Optimization, author = {Noorshams, Qais and Martens, Anne and Reussner, Ralf}, booktitle = {Proceedings of the 2nd International Workshop on the Quality of Service-Oriented Software Systems (QUASOSS '10), Oslo, Norway, October 4, 2010}, keywords = {t_workshop}, pages = {1:1–1:6}, publisher = {ACM, New York, NY, USA}, title = {Using Quality of Service Bounds for Effective Multi-objective Software Architecture Optimization}, year = 2010 }
• 
  Stochastic Performance Analysis and Capacity Planning of Publish/Subscribe Systems. A. Schröter; G. Mühl; S. Kounev; H. Parzyjegla; J. Richling; in 4th ACM International Conference on Distributed Event-Based Systems (DEBS 2010), July 12-15, Cambridge, United Kingdom (2010).

  Acceptance Rate: 25%
  • [ BibTeX ]
  • [ Download ]
  @inproceedings{SchroterMuhlRichling2010Stochastic, author = {Schr{{\"o}}ter, Arnd and M{{\"u}}hl, Gero and Kounev, Samuel and Parzyjegla, Helge and Richling, Jan}, booktitle = {4th ACM International Conference on Distributed Event-Based Systems (DEBS 2010), July 12-15, Cambridge, United Kingdom}, keywords = {Optimization}, note = {Acceptance Rate: 25\%}, publisher = {ACM, New York, USA}, title = {{Stochastic Performance Analysis and Capacity Planning of Publish/Subscribe Systems}}, year = 2010 }
• 
  Adaptive Packet Scheduling for Requests Delay Guaranties in Packet-Switched Communication Network. A. Grzech; P. ’S}}wika}tek; P. Rygielski; in Systems Science (2010). 36(1) 7–12.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this paper the problem of packet scheduling in the node of packet-switched computer communication network is considered. Packet scheduling in the network edge nodes is one of the crucial mechanisms essential for delivery of required level of quality of end-to-end network services (QoS). In order to satisfy QoS guaranties for each incoming request belonging to one of distinguished traffic classes, packet scheduling algorithm must make decisions based on current state of the scheduling system (e.g. buffer lengths) and actual characteristics of the serviced traffic (e.g.: lengths of connections following requests, packet intensities within requests, etc.). The paper is devoted to propose a new packet scheduling algorithm based on Weighted Round Robin (WRR), which weights are adapted according to changes of networkÂ’s load and traffic characteristics. By means of computer simulation, on representative examples, it was shown, that utilization of additional knowledge about incoming traffic in the process of packet scheduling may improve QoS guaranties for serviced traffic.

  @article{GrSwRy2010-SS, abstract = {In this paper the problem of packet scheduling in the node of packet-switched computer communication network is considered. Packet scheduling in the network edge nodes is one of the crucial mechanisms essential for delivery of required level of quality of end-to-end network services (QoS). In order to satisfy QoS guaranties for each incoming request belonging to one of distinguished traffic classes, packet scheduling algorithm must make decisions based on current state of the scheduling system (e.g. buffer lengths) and actual characteristics of the serviced traffic (e.g.: lengths of connections following requests, packet intensities within requests, etc.). The paper is devoted to propose a new packet scheduling algorithm based on Weighted Round Robin (WRR), which weights are adapted according to changes of networkÂ’s load and traffic characteristics. By means of computer simulation, on representative examples, it was shown, that utilization of additional knowledge about incoming traffic in the process of packet scheduling may improve QoS guaranties for serviced traffic.}, address = {Wroclaw, Poland}, author = {Grzech, Adam and {\'S}wi\k{a}tek, Pawel and Rygielski, Piotr}, journal = {Systems Science}, keywords = {t_journalmagazine}, number = 1, pages = {7–12}, publisher = {Wroclaw University of Technology Press}, title = {{Adaptive Packet Scheduling for Requests Delay Guaranties in Packet-Switched Communication Network}}, volume = 36, year = 2010 }
• 
  Performance Modeling in Industry: A Case Study on Storage Virtualization. N. Huber; S. Becker; C. Rathfelder; J. Schweflinghaus; R. Reussner; in ACM/IEEE 32nd International Conference on Software Engineering (ICSE 2010), Software Engineering in Practice Track (2010). 1–10.

  Acceptance Rate (Full Paper): 23% (16/71)
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In software engineering, performance and the integration of performance analysis methodologies gain increasing importance, especially for complex systems. Well-developed methods and tools can predict non-functional performance properties like response time or resource utilization in early design stages, thus promising time and cost savings. However, as performance modeling and performance prediction is still a young research area, the methods are not yet well-established and in wide-spread industrial use. This work is a case study of the applicability of the Palladio Component Model as a performance prediction method in an industrial environment. We model and analyze different design alternatives for storage virtualization on an IBM (Trademark of IBM in USA and/or other countries) system. The model calibration, validation and evaluation is based on data measured on a System z9 (Trademark of IBM in USA and/or other countries) as a proof of concept. The results show that performance predictions can identify performance bottlenecks and evaluate design alternatives in early stages of system development. The experiences gained were that performance modeling helps to understand and analyze a system. Hence, this case study substantiates that performance modeling is applicable in industry and a valuable method for evaluating design decisions.

  @inproceedings{HuBeRaScRe2010-ICSE-PerfMod, abstract = {In software engineering, performance and the integration of performance analysis methodologies gain increasing importance, especially for complex systems. Well-developed methods and tools can predict non-functional performance properties like response time or resource utilization in early design stages, thus promising time and cost savings. However, as performance modeling and performance prediction is still a young research area, the methods are not yet well-established and in wide-spread industrial use. This work is a case study of the applicability of the Palladio Component Model as a performance prediction method in an industrial environment. We model and analyze different design alternatives for storage virtualization on an IBM (Trademark of IBM in USA and/or other countries) system. The model calibration, validation and evaluation is based on data measured on a System z9 (Trademark of IBM in USA and/or other countries) as a proof of concept. The results show that performance predictions can identify performance bottlenecks and evaluate design alternatives in early stages of system development. The experiences gained were that performance modeling helps to understand and analyze a system. Hence, this case study substantiates that performance modeling is applicable in industry and a valuable method for evaluating design decisions.}, address = {New York, NY, USA}, author = {Huber, Nikolaus and Becker, Steffen and Rathfelder, Christoph and Schweflinghaus, Jochen and Reussner, Ralf}, booktitle = {ACM/IEEE 32nd International Conference on Software Engineering (ICSE 2010), Software Engineering in Practice Track}, keywords = {Automated_model_learning}, month = {05}, note = {Acceptance Rate (Full Paper): 23\% (16/71)}, pages = {1–10}, publisher = {ACM}, title = {{Performance Modeling in Industry: A Case Study on Storage Virtualization}}, year = 2010 }
• 
  METHOD AND SYSTEM FOR MANAGING LEARNING MATERIALS PRESENTED OFFLINE. R. Krebs; C. Hochwarth; (2010, April).
  • [ BibTeX ]
  @misc{Kr2010-uspatent-offline-content, author = {Krebs, Rouven and Hochwarth, Christian}, howpublished = {Patent US8644755}, keywords = {se2}, month = {04}, title = {METHOD AND SYSTEM FOR MANAGING LEARNING MATERIALS PRESENTED OFFLINE}, year = 2010 }
• 
  Graph-fold: an Efficient Method for Complex Service Execution Plan Optimization. P. Rygielski; P. ’S}}wika}tek; in Systems Science (2010). 36(3) 25–32.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The task of optimization of the complex service execution plan is the last stage of the process of complex services composition in systems based on the SOA paradigm. In general this task consists of choosing proper versions of atomic services, delivering requested functionalities, such that non-functional requirements for complex service are satisfied. In most real-life cases this task is reduced to the problem of multidimensional knapsack problem (MKP), which is known to be NP-hard. Therefore, in order to facilitate effective complex service composition, it is crucial to provide an efficient MKP solution algorithm. In this paper a novel approach to the problem of optimization of complex service execution plan is introduced. In the proposed approach the MKP feasible solution space is constructed in such a way, that the optimal solution can be found in constant time. Of course, since MKP is NP-hard, the process of solution space construction takes an exponential time, This time, however, is amortized by multiple usages of single solution space for determination of optimal execution plans for multiple complex service requests.

  @article{RySw2010-SS, abstract = {The task of optimization of the complex service execution plan is the last stage of the process of complex services composition in systems based on the SOA paradigm. In general this task consists of choosing proper versions of atomic services, delivering requested functionalities, such that non-functional requirements for complex service are satisfied. In most real-life cases this task is reduced to the problem of multidimensional knapsack problem (MKP), which is known to be NP-hard. Therefore, in order to facilitate effective complex service composition, it is crucial to provide an efficient MKP solution algorithm. In this paper a novel approach to the problem of optimization of complex service execution plan is introduced. In the proposed approach the MKP feasible solution space is constructed in such a way, that the optimal solution can be found in constant time. Of course, since MKP is NP-hard, the process of solution space construction takes an exponential time, This time, however, is amortized by multiple usages of single solution space for determination of optimal execution plans for multiple complex service requests.}, address = {Wroclaw, Poland}, author = {Rygielski, Piotr and {\'S}wi\k{a}tek, Pawel}, journal = {Systems Science}, keywords = {Optimization}, number = 3, pages = {25–32}, publisher = {Wroclaw University of Technology Press}, title = {{Graph-fold: an Efficient Method for Complex Service Execution Plan Optimization}}, volume = 36, year = 2010 }
• 
  Engineering of Next Generation Self-Aware Software Systems: A Research Roadmap. S. Kounev; in Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe (2010).
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  @inproceedings{Ko2010-KIT-Roadmap, address = {Karlsruhe, Germany}, author = {Kounev, Samuel}, booktitle = {{Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe}}, keywords = {t_workshop}, month = {07}, publisher = {KIT Scientific Publishing}, title = {{Engineering of Next Generation Self-Aware Software Systems: A Research Roadmap}}, year = 2010 }
• 
  Wireless Network Management Through Network-Enforced Handover. P. Rygielski; P. ’S}}wika}tek; in Applications of Systems Science, A. Grzech, P. ’S}}wika}tek, K. Brzostowski (eds.) (2010). 227–236.
  • [ BibTeX ]
  @incollection{RyPs2010-ICSS-handover, address = {ul. Maszynowa 5/9, 02-392 Warszawa, Poland}, author = {Rygielski, Piotr and {\'S}wi\k{a}tek, Pawe{\l}{}}, booktitle = {Applications of Systems Science}, editor = {Grzech, Adam and {\'S}wi\k{a}tek, Pawel and Brzostowski, Krzysztof}, keywords = {Optimization}, month = {09}, pages = {227–236}, publisher = {Exit}, title = {{Wireless Network Management Through Network-Enforced Handover}}, year = 2010 }
• 
  Performance Management in the Cloud. S. Kounev; (2010, July).
  • [ BibTeX ]
  @misc{talk-Kounev-GIF10, author = {Kounev, Samuel}, howpublished = {Invited Talk, Annual Meeting of the GI Fachgruppe Software Architectures 2010, Munich, Germany}, keywords = {Talks}, month = {07}, title = {{Performance Management in the Cloud}}, year = 2010 }
• 
  Translations of Service Level Agreement in Systems Based on Service Oriented Architectures. A. Grzech; P. ’S}}wika}tek; P. Rygielski; in Cybernetics and Systems: An International Journal (CBS) (2010). 41(8) 610–627.

  5-Year Impact Factor (2014): 0.968
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The gain of the article is to introduce and to discuss a formal specification of a computer system's service level agreement (SLA) and its translation into the structure of complex services (composed of atomic services) delivering required functionalities and nonfunctionalities in a distributed environment. It is assumed that the SLA is composed of two parts specifying quantitative and qualitative requirements. The former requirements define the structure of the adequate complex services in form of a directed graph, where potential parallelism of atomic services performance may be taken into account. The latter qualitative requirements are applied to select the optimal complex service realization scenario; it is based on assumption that various atomic services distinguished in the complex services structures are available at the considered distributed environment in different versions and locations. Different versions of atomic services are atomic services delivering the required functionalities and satisfy non-functionalities at various levels. Different locations (installation place) of available atomic services means that the cost of atomic services delivery (communication and calculation) depends on parts of the distributed systems where the services are performed. The proposed model of SLA translation into complex services structures and variants may be applied among others to calculate upper and lower complex services delivery times and to estimate the validity of possible parallelism in complex services.

  @article{GrSwRy2010-CBS, abstract = {The gain of the article is to introduce and to discuss a formal specification of a computer system's service level agreement (SLA) and its translation into the structure of complex services (composed of atomic services) delivering required functionalities and nonfunctionalities in a distributed environment. It is assumed that the SLA is composed of two parts specifying quantitative and qualitative requirements. The former requirements define the structure of the adequate complex services in form of a directed graph, where potential parallelism of atomic services performance may be taken into account. The latter qualitative requirements are applied to select the optimal complex service realization scenario; it is based on assumption that various atomic services distinguished in the complex services structures are available at the considered distributed environment in different versions and locations. Different versions of atomic services are atomic services delivering the required functionalities and satisfy non-functionalities at various levels. Different locations (installation place) of available atomic services means that the cost of atomic services delivery (communication and calculation) depends on parts of the distributed systems where the services are performed. The proposed model of SLA translation into complex services structures and variants may be applied among others to calculate upper and lower complex services delivery times and to estimate the validity of possible parallelism in complex services.}, address = {Bristol, PA, USA}, author = {Grzech, Adam and {\'S}wi\k{a}tek, Pawel and Rygielski, Piotr}, journal = {{Cybernetics and Systems: An International Journal (CBS)}}, keywords = {t_journalmagazine}, month = 11, note = {5-Year Impact Factor (2014): 0.968}, number = 8, pages = {610–627}, publisher = {Taylor & Francis, Inc.}, title = {Translations of Service Level Agreement in Systems Based on Service Oriented Architectures}, volume = 41, year = 2010 }
• 
  A Framework for Multi-level SLA Management. M. Comuzzi; C. Kotsokalis; C. Rathfelder; W. Theilmann; U. Winkler; G. Zacco; in Service-Oriented Computing. ICSOC/ServiceWave 2009 Workshops, A. Dan, F. Gittler, F. Toumani (eds.) (2010). (Vol. 6275) 187–196.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Service-Oriented Architectures (SOA) represent an architectural shift for building business applications based on loosely-coupled services. In a multi-layered SOA environment the exact conditions under which services are to be delivered can be formally specified by Service Level Agreements (SLAs). However, typical SLAs are just specified at the customer-level and do not allow service providers to manage their IT stack accordingly as they have no insight on how customer-level SLAs translate to metrics or parameters at the various layers of the IT stack. In this paper we present a technical architecture for a multi-level SLA management framework. We discuss the fundamental components and in- terfaces in this architecture and explain the developed integrated framework. Furthermore, we show results from a qualitative evaluation of the framework in the context of an open reference case.

  @inproceedings{comuzzi2009, abstract = {Service-Oriented Architectures (SOA) represent an architectural shift for building business applications based on loosely-coupled services. In a multi-layered SOA environment the exact conditions under which services are to be delivered can be formally specified by Service Level Agreements (SLAs). However, typical SLAs are just specified at the customer-level and do not allow service providers to manage their IT stack accordingly as they have no insight on how customer-level SLAs translate to metrics or parameters at the various layers of the IT stack. In this paper we present a technical architecture for a multi-level SLA management framework. We discuss the fundamental components and in- terfaces in this architecture and explain the developed integrated framework. Furthermore, we show results from a qualitative evaluation of the framework in the context of an open reference case.}, address = {Berlin, Heidelberg}, author = {Comuzzi, Marco and Kotsokalis, Constantinos and Rathfelder, Christoph and Theilmann, Wolfgang and Winkler, Ulrich and Zacco, Gabriele}, booktitle = {Service-Oriented Computing. ICSOC/ServiceWave 2009 Workshops}, editor = {Dan, Asit and Gittler, Fr{\'e}d{\'e}ric and Toumani, Farouk}, keywords = {se2}, month = 11, pages = {187–196}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {A Framework for Multi-level SLA Management}, volume = 6275, year = 2010 }
• 
  Dynamic Resources Allocation for Delivery of Personalized Services. A. Grzech; P. ’S}}wika}tek; P. Rygielski; in Software Services for e-World, W. Cellary, E. Estevez (eds.) (2010). (Vol. 341) 17–28.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The aim of this paper is to introduce a problem of e-health services quality management. The process of delivering e-health services to users consists of two major tasks: service personalization and resources allocation. In this paper we introduce a use-cases of e-health system and distinguish services that can be offered. In order to satisfy user requirements one have to manage resources properly especially when communication conditions change (e.g. in an ambulance). An exemplary solution has been presented and conclusions for further work have been formulated.

  @incollection{GrSwRy2010-IFIP, abstract = {The aim of this paper is to introduce a problem of e-health services quality management. The process of delivering e-health services to users consists of two major tasks: service personalization and resources allocation. In this paper we introduce a use-cases of e-health system and distinguish services that can be offered. In order to satisfy user requirements one have to manage resources properly especially when communication conditions change (e.g. in an ambulance). An exemplary solution has been presented and conclusions for further work have been formulated.}, address = {Boston}, author = {Grzech, Adam and {\'S}wi\k{a}tek, Pawel and Rygielski, Piotr}, booktitle = {Software Services for e-World}, editor = {Cellary, Wojciech and Estevez, Elsa}, keywords = {Virtualization}, month = 11, pages = {17–28}, publisher = {Springer}, series = {IFIP Advances in Information and Communication Technology}, title = {{Dynamic Resources Allocation for Delivery of Personalized Services}}, volume = 341, year = 2010 }
• 
  Interval Comparisons and Lattice Operations based on the Interval Overlapping Relation. (476), M. Nehmeier; J. Wolff von Gudenberg; (2010).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper we start with the definition of the interval overlapping relation showing the 13 different cases that may occur when to compare two non-empty intervals. This paper serves 2 purposes: 1. to define a solid foundation for interval comparisons and lattice operations 2. to provide an advanced interface for language embedding and application programs.

  @techreport{nehmeierTR476, abstract = {In this paper we start with the definition of the interval overlapping relation showing the 13 different cases that may occur when to compare two non-empty intervals. This paper serves 2 purposes: 1. to define a solid foundation for interval comparisons and lattice operations 2. to provide an advanced interface for language embedding and application programs.}, author = {Nehmeier, Marco and Wolff von Gudenberg, J{\"u}rgen}, institution = {Department of Computer Science, University of W{\"u}rzburg}, keywords = {Reliability}, month = {09}, number = 476, title = {{I}nterval {C}omparisons and {L}attice {O}perations based on the {I}nterval {O}verlapping {R}elation}, type = {Technical Report}, year = 2010 }
• 
  Predictive Modelling of Peer-to-Peer Event-driven Communication in Component-based Systems. C. Rathfelder; D. Evans; S. Kounev; in Proceedings of the 7th European Performance Engineering Workshop (EPEW 2010), A. Aldini, M. Bernardo, L. Bononi, V. Cortellessa (eds.) (2010). (Vol. 6342) 219–235.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The event-driven communication paradigm is used increasingly often to build loosely-coupled distributed systems in many industry domains including telecommunications, transportation, and supply chain management. However, the loose coupling of components in such systems makes it hard for developers to estimate their behaviour and performance under load. Most general purpose performance meta-models for component-based systems provide limited support for modelling event-driven communication. In this paper, we present a case study of a real-life road traffic monitoring system that shows how event-driven communication can be modelled for performance prediction and capacity planning. Our approach is based on the Palladio Component Model (PCM) which we have extended to support event-driven communication. We evaluate the accuracy of our modelling approach in a number of different workload and configuration scenarios. The results demonstrate the practicality and effectiveness of the proposed approach.

  @inproceedings{rathfelder2010a, abstract = {The event-driven communication paradigm is used increasingly often to build loosely-coupled distributed systems in many industry domains including telecommunications, transportation, and supply chain management. However, the loose coupling of components in such systems makes it hard for developers to estimate their behaviour and performance under load. Most general purpose performance meta-models for component-based systems provide limited support for modelling event-driven communication. In this paper, we present a case study of a real-life road traffic monitoring system that shows how event-driven communication can be modelled for performance prediction and capacity planning. Our approach is based on the Palladio Component Model (PCM) which we have extended to support event-driven communication. We evaluate the accuracy of our modelling approach in a number of different workload and configuration scenarios. The results demonstrate the practicality and effectiveness of the proposed approach.}, address = {Berlin, Heidelberg}, author = {Rathfelder, Christoph and Evans, David and Kounev, Samuel}, booktitle = {Proceedings of the 7th European Performance Engineering Workshop (EPEW 2010)}, editor = {Aldini, Alessandro and Bernardo, Marco and Bononi, Luciano and Cortellessa, Vittorio}, keywords = {t_workshop}, month = {09}, pages = {219–235}, publisher = {Springer-Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{P}redictive {M}odelling of {P}eer-to-{P}eer {E}vent-driven {C}ommunication in {C}omponent-based {S}ystems}, volume = 6342, year = 2010 }
• 
  Analysis of the Performance-Influencing Factors of Virtualization Platforms. N. Huber; M. von Quast; F. Brosig; S. Kounev; in The 12th International Symposium on Distributed Objects, Middleware, and Applications (DOA 2010) (2010).

  Acceptance Rate (Full Paper): 33%
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Nowadays, virtualization solutions are gaining increasing importance. By enabling the sharing of physical resources, thus making resource usage more efficient, they promise energy and cost savings. Additionally, virtualization is the key enabling technology for Cloud Computing and server consolidation. However, the effects of sharing resources on system performance are not yet well-understood. This makes performance prediction and performance management of services deployed in such dynamic systems very challenging. Because of the large variety of virtualization solutions, a generic approach to predict the performance influences of virtualization platforms is highly desirable. In this paper, we present a hierarchical model capturing the major performance-relevant factors of virtualization platforms. We then propose a general methodology to quantify the influence of the identified factors based on an empirical approach using benchmarks. Finally, we present a case study of Citrix XenServer 5.5, a state-of-the-art virtualization platform.

  @inproceedings{HuQuBrKo2010-DOA-AnalysisVirt, abstract = {Nowadays, virtualization solutions are gaining increasing importance. By enabling the sharing of physical resources, thus making resource usage more efficient, they promise energy and cost savings. Additionally, virtualization is the key enabling technology for Cloud Computing and server consolidation. However, the effects of sharing resources on system performance are not yet well-understood. This makes performance prediction and performance management of services deployed in such dynamic systems very challenging. Because of the large variety of virtualization solutions, a generic approach to predict the performance influences of virtualization platforms is highly desirable. In this paper, we present a hierarchical model capturing the major performance-relevant factors of virtualization platforms. We then propose a general methodology to quantify the influence of the identified factors based on an empirical approach using benchmarks. Finally, we present a case study of Citrix XenServer 5.5, a state-of-the-art virtualization platform.}, address = {Crete, Greece}, author = {Huber, Nikolaus and von Quast, Marcel and Brosig, Fabian and Kounev, Samuel}, booktitle = {The 12th International Symposium on Distributed Objects, Middleware, and Applications (DOA 2010)}, keywords = {Virtualization}, month = 10, note = {Acceptance Rate (Full Paper): 33\%}, publisher = {Springer Verlag}, title = {{Analysis of the Performance-Influencing Factors of Virtualization Platforms}}, year = 2010 }
• 
  Optimal Complex Services Composition in SOA Systems. P. Rygielski; P. ’S}}wika}tek; in Applications of Systems Science, A. Grzech, P. ’S}}wika}tek, K. Brzostowski (eds.) (2010). 217–226.

  Most Promising Student Paper Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  One of the most important tasks in service oriented architecture paradigm based systems is the task of composition of the complex service. Aim of this paper is to focus on the very last phase of the composition process, where proper atomic service versions are picked to execute the whole complex service. An exact algorithm is proposed in this paper solving two tasks based on popular quality of service delivery approaches: best-effort and differentiated services. The procedure of graph reduction has been introduced to generate space of possible solutions in such way that decision is made in more efficient way. The graph-fold algorithm was tested in simulation environment where its performance was compared to five reference algorithms. Moreover ability to deliver quality at desired level has been tested.

  @incollection{RyPs2010-ICSS-GF, abstract = {One of the most important tasks in service oriented architecture paradigm based systems is the task of composition of the complex service. Aim of this paper is to focus on the very last phase of the composition process, where proper atomic service versions are picked to execute the whole complex service. An exact algorithm is proposed in this paper solving two tasks based on popular quality of service delivery approaches: best-effort and differentiated services. The procedure of graph reduction has been introduced to generate space of possible solutions in such way that decision is made in more efficient way. The graph-fold algorithm was tested in simulation environment where its performance was compared to five reference algorithms. Moreover ability to deliver quality at desired level has been tested.}, address = {ul. Maszynowa 5/9, 02-392 Warszawa, Poland}, author = {Rygielski, Piotr and {\'S}wi\k{a}tek, Pawe{\l}{}}, booktitle = {Applications of Systems Science}, editor = {Grzech, Adam and {\'S}wi\k{a}tek, Pawel and Brzostowski, Krzysztof}, keywords = {Optimization}, month = {09}, note = {<b>Most Promising Student Paper Award</b>}, pages = {217–226}, publisher = {Exit}, title = {{Optimal Complex Services Composition in SOA Systems}}, year = 2010 }
• 
  Parametric Performance Completions for Model-Driven Performance Prediction. J. Happe; S. Becker; C. Rathfelder; H. Friedrich; R. H. Reussner; in Performance Evaluation (PE) (2010). 67(8) 694–716.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Performance prediction methods can help software architects to identify potential performance problems, such as bottlenecks, in their software systems during the design phase. In such early stages of the software life-cycle, only a little information is available about the system?s implementation and execution environment. However, these details are crucial for accurate performance predictions. Performance completions close the gap between available high-level models and required low-level details. Using model-driven technologies, transformations can include details of the implementation and execution environment into abstract performance models. However, existing approaches do not consider the relation of actual implementations and performance models used for prediction. Furthermore, they neglect the broad variety of possible implementations and middleware platforms, possible configurations, and possible usage scenarios. In this paper, we (i) establish a formal relation between generated performance models and generated code, (ii) introduce a design and application process for parametric performance completions, and (iii) develop a parametric performance completion for Message-oriented Middleware according to our method. Parametric performance completions are independent of a specific platform, reflect performance-relevant software configurations, and capture the influence of different usage scenarios. To evaluate the prediction accuracy of the completion for Message-oriented Middleware, we conducted a real-world case study with the SPECjms2007 Benchmark [http://www.spec.org/jms2007/]. The observed deviation of measurements and predictions was below 10% to 15%

  @article{happe2009a, abstract = {Performance prediction methods can help software architects to identify potential performance problems, such as bottlenecks, in their software systems during the design phase. In such early stages of the software life-cycle, only a little information is available about the system?s implementation and execution environment. However, these details are crucial for accurate performance predictions. Performance completions close the gap between available high-level models and required low-level details. Using model-driven technologies, transformations can include details of the implementation and execution environment into abstract performance models. However, existing approaches do not consider the relation of actual implementations and performance models used for prediction. Furthermore, they neglect the broad variety of possible implementations and middleware platforms, possible configurations, and possible usage scenarios. In this paper, we (i) establish a formal relation between generated performance models and generated code, (ii) introduce a design and application process for parametric performance completions, and (iii) develop a parametric performance completion for Message-oriented Middleware according to our method. Parametric performance completions are independent of a specific platform, reflect performance-relevant software configurations, and capture the influence of different usage scenarios. To evaluate the prediction accuracy of the completion for Message-oriented Middleware, we conducted a real-world case study with the SPECjms2007 Benchmark [http://www.spec.org/jms2007/]. The observed deviation of measurements and predictions was below 10% to 15%}, author = {Happe, Jens and Becker, Steffen and Rathfelder, Christoph and Friedrich, Holger and Reussner, Ralf H.}, journal = {Performance Evaluation (PE)}, keywords = {se2}, month = {08}, number = 8, pages = {694–716}, publisher = {Elsevier}, title = {{P}arametric {P}erformance {C}ompletions for {M}odel-{D}riven {P}erformance {P}rediction}, volume = 67, year = 2010 }
• 
  Towards Self-Aware Dependability Management in Virtualized Service Infrastructures. S. Kounev; (2010, July).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Dagstuhl10, author = {Kounev, Samuel}, howpublished = {Dagstuhl Seminar 10292, Resilience Assessment and Evaluation, Schloss Dagstuhl, Germany}, keywords = {Talks}, month = {07}, title = {{Towards Self-Aware Dependability Management in Virtualized Service Infrastructures}}, year = 2010 }
• 
  Towards self-aware performance and resource management in modern service-oriented systems. S. Kounev; F. Brosig; N. Huber; R. Reussner; in Proceedings of the 7th IEEE International Conference on Services Computing (SCC 2010), July 5-10, Miami, Florida, USA (2010).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Modern service-oriented systems have increasingly complex loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment (e.g., varying service workloads) and adapt the system configuration accordingly. In this paper, we describe a long-term vision and approach for designing systems with built-in self-aware performance and resource management capabilities. We advocate the use of architecture-level performance models extracted dynamically from the evolving system configuration and maintained automatically during operation. The models will be exploited at run-time to adapt the system to changes in the environment ensuring that resources are utilized efficiently and performance requirements are continuously satisfied.

  @inproceedings{KoBrHuRe2010-SCC-Towards, abstract = {Modern service-oriented systems have increasingly complex loosely-coupled architectures that often exhibit poor performance and resource efficiency and have high operating costs. This is due to the inability to predict at run-time the effect of dynamic changes in the system environment (e.g., varying service workloads) and adapt the system configuration accordingly. In this paper, we describe a long-term vision and approach for designing systems with built-in self-aware performance and resource management capabilities. We advocate the use of architecture-level performance models extracted dynamically from the evolving system configuration and maintained automatically during operation. The models will be exploited at run-time to adapt the system to changes in the environment ensuring that resources are utilized efficiently and performance requirements are continuously satisfied.}, author = {Kounev, Samuel and Brosig, Fabian and Huber, Nikolaus and Reussner, Ralf}, booktitle = {Proceedings of the 7th IEEE International Conference on Services Computing (SCC 2010), July 5-10, Miami, Florida, USA}, keywords = {Self-adaptive-systems}, month = {07}, publisher = {IEEE Computer Society}, title = {{Towards self-aware performance and resource management in modern service-oriented systems}}, year = 2010 }
• 
  Towards Middleware-Aware Integration of Event-based Communication into the Palladio Component Model. C. Rathfelder; B. Klatt; S. Kounev; D. Evans; in Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems (DEBS 2010) (2010). 97–98.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The event-based communication paradigm is becoming increasingly ubiquitous as an enabling technology for building loosely-coupled distributed systems. However, the loose coupling of components in such systems makes it hard for developers to predict their performance under load. Most general purpose performance meta-models for component-based systems provide limited support for modelling event-based communication and neglect middleware-specific influence factors. In this poster, we present an extension of our approach to modelling event-based communication in the context of the Palladio Component Model (PCM), allowing to take into account middleware-specific influence factors. The latter are captured in a separate model automatically woven into the PCM instance by means of a model-to-model transformation. As a second contribution, we present a short case study of a real-life road traffic monitoring system showing how event-based communication can be modelled for performance prediction and capacity planning.

  @inproceedings{rathfelder2010b, abstract = {The event-based communication paradigm is becoming increasingly ubiquitous as an enabling technology for building loosely-coupled distributed systems. However, the loose coupling of components in such systems makes it hard for developers to predict their performance under load. Most general purpose performance meta-models for component-based systems provide limited support for modelling event-based communication and neglect middleware-specific influence factors. In this poster, we present an extension of our approach to modelling event-based communication in the context of the Palladio Component Model (PCM), allowing to take into account middleware-specific influence factors. The latter are captured in a separate model automatically woven into the PCM instance by means of a model-to-model transformation. As a second contribution, we present a short case study of a real-life road traffic monitoring system showing how event-based communication can be modelled for performance prediction and capacity planning.}, address = {New York, NY, USA}, author = {Rathfelder, Christoph and Klatt, Benjamin and Kounev, Samuel and Evans, David}, booktitle = {Proceedings of the Fourth ACM International Conference on Distributed Event-Based Systems (DEBS 2010)}, keywords = {t_poster}, month = {07}, pages = {97–98}, publisher = {ACM}, title = {{Towards Middleware-Aware Integration of Event-based Communication into the Palladio Component Model}}, year = 2010 }
• 
  Engineering of Next Generation Self-Aware Software Systems: A Research Roadmap. S. Kounev; (2010, July).
  • [ BibTeX ]
  @misc{talk-Kounev-KITNachwuchs10, author = {Kounev, Samuel}, howpublished = {Informatik am KIT: Die Nachwuchsgruppen stellen sich vor, Karlsruhe Institute of Technology, Karlsruhe, Germany}, keywords = {Talks}, month = {07}, title = {{Engineering of Next Generation Self-Aware Software Systems: A Research Roadmap}}, year = 2010 }
• 
  Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe S. Kounev; (V. Pankratius; S. Kounev, eds.) (2010). KIT Scientific Publishing, Karlsruhe, Germany.

  ISBN: 978-3-86644-508-6
  • [ BibTeX ]
  • [ URL ]
  @proceedings{PaKo2010-KIT-YoungInformatics, address = {Karlsruhe, Germany}, author = {Kounev, Samuel}, editor = {Pankratius, Victor and Kounev, Samuel}, keywords = {t_workshop}, month = {07}, note = {ISBN: 978-3-86644-508-6}, publisher = {KIT Scientific Publishing}, title = {{Emerging Research Directions in Computer Science. Contributions from the Young Informatics Faculty in Karlsruhe}}, year = 2010 }
• 
  Translations of Service Level Agreement in Systems Based on Service Oriented Architecture. A. Grzech; P. Rygielski; in Proceedings of the 14th International Conference on Knowledge-based and Intelligent Information and Engineering Systems (KES 2010), R. Setchi, I. Jordanov (eds.) (2010). (Vol. 6277) 523–532.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The gain of the paper is to introduce and to discuss a formal specification of computer system's Service Level Agreement (SLA) and its translation into structure of complex services (composed of atomic services) delivering required functionalities and non-functionalities in distributed environment. The SLA is composed of two parts specifying quantitative and qualitative requirements. The former requirements define structure of the adequate complex services as a directed graph, where potential parallelism of atomic services performance may be taken into account. The qualitative requirements are applied to select the optimal complex service realization scenario; it is based on assumption that various atomic services distinguished in the complex services structure are available in the environment in different versions and locations. Different versions of atomic services are delivering the required functionalities and satisfy non-functionalities at various levels. Various locations of atomic services means that the time and cost of atomic services delivery (communication and calculation) may vary. Proposed model of SLA translation into complex services causes that a scenario variants may be applied, among others, to calculate upper and lower complex services delivering times and to estimate validity of possible parallelism in complex services.

  @inproceedings{GrRy2010-KES, abstract = {The gain of the paper is to introduce and to discuss a formal specification of computer system's Service Level Agreement (SLA) and its translation into structure of complex services (composed of atomic services) delivering required functionalities and non-functionalities in distributed environment. The SLA is composed of two parts specifying quantitative and qualitative requirements. The former requirements define structure of the adequate complex services as a directed graph, where potential parallelism of atomic services performance may be taken into account. The qualitative requirements are applied to select the optimal complex service realization scenario; it is based on assumption that various atomic services distinguished in the complex services structure are available in the environment in different versions and locations. Different versions of atomic services are delivering the required functionalities and satisfy non-functionalities at various levels. Various locations of atomic services means that the time and cost of atomic services delivery (communication and calculation) may vary. Proposed model of SLA translation into complex services causes that a scenario variants may be applied, among others, to calculate upper and lower complex services delivering times and to estimate validity of possible parallelism in complex services.}, address = {Berlin, Heidelberg}, author = {Grzech, Adam and Rygielski, Piotr}, booktitle = {Proceedings of the 14th International Conference on Knowledge-based and Intelligent Information and Engineering Systems (KES 2010)}, editor = {Setchi, Rossitza and Jordanov, Ivan}, keywords = {descartes}, month = {09}, pages = {523–532}, publisher = {Springer-Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Translations of Service Level Agreement in Systems Based on Service Oriented Architecture}}, volume = 6277, year = 2010 }

• 
  Best of Eclipse DemoCamps - Ein Erfahrungsbericht vom dritten Karlsruher Eclipse DemoCamp. H. Groenda; C. Rathfelder; R. Mueller; in Eclipse Magazine (2009). 3 8–10.
  • [ BibTeX ]
  @article{groenda2009, author = {Groenda, Henning and Rathfelder, Christoph and Mueller, Ralph}, journal = {Eclipse Magazine}, keywords = {se2}, month = {03}, pages = {8–10}, title = {{Best of Eclipse DemoCamps - Ein Erfahrungsbericht vom dritten Karlsruher Eclipse DemoCamp}}, volume = 3, year = 2009 }
• 
  SOA2 - A Vision of Next Generation Service-Oriented Architectures. S. Kounev; (2009, April).
  • [ BibTeX ]
  @misc{talk-Kounev-ABB09, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), ABB Corporate Research, Ladenburg, Germany}, keywords = {Talks}, month = {04}, title = {{SOA2 - A Vision of Next Generation Service-Oriented Architectures}}, year = 2009 }
• 
  Autonomic QoS control in enterprise Grid environments using online simulation. R. Nou; S. Kounev; F. Julia; J. Torres; in Journal of Systems and Software (2009). 82(3) 486–502.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.

  @article{NoKoJuTo2008-JSS-GridAutoQoS, abstract = {As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, a comprehensive framework for autonomic QoS control in enterprise Grid environments using online simulation is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. Support for advanced features such as autonomic workload characterization on-the-fly, dynamic deployment of Grid servers on demand, as well as dynamic system reconfiguration after a server failure is provided. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.}, address = {Amsterdam, The Netherlands}, author = {Nou, Ramon and Kounev, Samuel and Julia, Ferran and Torres, Jordi}, journal = {Journal of Systems and Software}, keywords = {Self-adaptive-systems}, month = {03}, number = 3, pages = {486–502}, publisher = {Elsevier Science Publishers B. V.}, title = {{Autonomic QoS control in enterprise Grid environments using online simulation}}, volume = 82, year = 2009 }
• 
  QPME - A Performance Modeling Tool Based on Queueing Petri Nets. S. Kounev; C. Dutz; in ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis (2009). 36 46–51.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.

  @article{KoDu2009-SIGMETRICS_PER-QPME, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. By combining the modeling power and expressiveness of queueing networks and stochastic Petri nets, queueing Petri nets provide a number of advantages. In this paper, we present QPME (Queueing Petri net Modeling Environment) - a tool that supports the modeling and analysis of systems using queueing Petri nets. QPME provides an Eclipse-based editor for designing queueing Petri net models and a powerful simulation engine for analyzing the models. After presenting the tool, we discuss the ongoing work on the QPME project and the planned future enhancements of the tool.}, author = {Kounev, Samuel and Dutz, Christofer}, journal = {ACM SIGMETRICS Performance Evaluation Review (PER), Special Issue on Tools for Computer Performance Modeling and Reliability Analysis}, keywords = {QPN}, month = {03}, pages = {46–51}, publisher = {ACM}, title = {{QPME - A Performance Modeling Tool Based on Queueing Petri Nets}}, volume = 36, year = 2009 }
• 
  Stochastic Analysis of Hierarchical Publish/Subscribe Systems. G. Mühl; A. Schröter; H. Parzyjegla; S. Kounev; J. Richling; in Proceedings of the 15th International European Conference on Parallel and Distributed Computing (Euro-Par 2009), Delft, The Netherlands, August 25-28, 2009. (2009).

  Acceptance Rate (Full Paper): 33%
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  With the gradual adoption of publish/subscribe systems in mission critical areas, it is essential that systems are subjected to rigorous performance analysis before they are put into production. However, existing approaches to performance modeling and analysis of publish/subscribe systems suffer from many limitations that seriously constrain their practical applicability. In this paper, we present a generalized method for stochastic analysis of publish/subscribe systems employing identity-based hierarchical routing. The method is based on an analytical model that addresses the major limitations underlying existing work in this area. In particular, it supports arbitrary broker overlay topologies and allows to set workload parameters, e.g., publication rates and subscription lifetimes, individually for each broker. The analysis is illustrated by a running example that helps to gain better understanding of the derived mathematical relationships.

  @inproceedings{MuScPaKoRi2009-EuroPar-StoAnalPubSub, abstract = {With the gradual adoption of publish/subscribe systems in mission critical areas, it is essential that systems are subjected to rigorous performance analysis before they are put into production. However, existing approaches to performance modeling and analysis of publish/subscribe systems suffer from many limitations that seriously constrain their practical applicability. In this paper, we present a generalized method for stochastic analysis of publish/subscribe systems employing identity-based hierarchical routing. The method is based on an analytical model that addresses the major limitations underlying existing work in this area. In particular, it supports arbitrary broker overlay topologies and allows to set workload parameters, e.g., publication rates and subscription lifetimes, individually for each broker. The analysis is illustrated by a running example that helps to gain better understanding of the derived mathematical relationships.}, author = {M{\"u}hl, Gero and Schr{\"o}ter, Arnd and Parzyjegla, Helge and Kounev, Samuel and Richling, Jan}, booktitle = {Proceedings of the 15th International European Conference on Parallel and Distributed Computing (Euro-Par 2009), Delft, The Netherlands, August 25-28, 2009.}, keywords = {Optimization}, note = {Acceptance Rate (Full Paper): 33%}, publisher = {Springer Verlag}, title = {{Stochastic Analysis of Hierarchical Publish/Subscribe Systems}}, year = 2009 }
• 
  Keynote: SOA2 - Eine Vision für die nächste Generation dienstorienterter Software-Architekturen. S. Kounev; (2009, February).
  • [ BibTeX ]
  @misc{talk-Kounev-SOA209, author = {Kounev, Samuel}, howpublished = {Invited Talk, Deutschepost Strategietagung - 6100 Application Service Management MAIL/Retail, St. Goar, Germany}, keywords = {Talks}, month = {02}, title = {{Keynote: SOA2 - Eine Vision f\"{u}r die n\"{a}chste Generation dienstorienterter Software-Architekturen}}, year = 2009 }
• 
  Software Performance Evaluation. S. Kounev; in Wiley Encyclopedia of Computer Science and Engineering, edited by Benjamin W. Wah (2009).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modern software systems are expected to satisfy increasingly stringent requirements for performance and scalability. To avoid the pitfalls of inadequate quality of service, it is important to evaluate the expected performance and scalability characteristics of systems during all phases of their life cycle. At every stage, performance evaluation is carried out with a specific set of goals and constraints. In this article, we present an overview of the major methods and techniques for software performance evaluation. We start by considering the different types of workload models that are typically used in performance evaluation studies. We then discuss performance measurement techniques including platform benchmarking, application profiling and system load testing. Following this, we survey the most common methods and techniques for performance modeling of software systems. We consider the major types of performance models used in practice and discuss their advantages and disadvantages. Finally, we briefly discuss operational analysis as an alternative to queueing theoretic methods.

  @incollection{Ko2008-WILEY-SoftwarePerfEval, abstract = {Modern software systems are expected to satisfy increasingly stringent requirements for performance and scalability. To avoid the pitfalls of inadequate quality of service, it is important to evaluate the expected performance and scalability characteristics of systems during all phases of their life cycle. At every stage, performance evaluation is carried out with a specific set of goals and constraints. In this article, we present an overview of the major methods and techniques for software performance evaluation. We start by considering the different types of workload models that are typically used in performance evaluation studies. We then discuss performance measurement techniques including platform benchmarking, application profiling and system load testing. Following this, we survey the most common methods and techniques for performance modeling of software systems. We consider the major types of performance models used in practice and discuss their advantages and disadvantages. Finally, we briefly discuss operational analysis as an alternative to queueing theoretic methods.}, author = {Kounev, Samuel}, booktitle = {{Wiley Encyclopedia of Computer Science and Engineering, edited by Benjamin W. Wah}}, keywords = {t_bookchapter}, month = {01}, publisher = {Wiley-Interscience, John Wiley & Sons Inc.}, title = {{Software Performance Evaluation}}, year = 2009 }
• 
  Effiziente Intervallarithmetik mit C++. M. Nehmeier; Thesis; Lehrstuhl für Informatik 2, Universität Würzburg. (2009).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Im Bereich des wissenschaftlichen Rechnens stösst die Intervallarithmetik in den letzten Jahren auf immer mehr Akzeptanz. Eigenschaften, wie das sichere Einschliessen der Ergebnisse bei der Intervallauswertung, machen sie zu einem guten Mittel um mit den Rundungsfehlern der Gleitkommazahlen umzugehen. Auch können Berechnungen auf Wertemengen mit garantierten Grenzen durchgeführt werden. Ebenso wird in Anwendungsgebieten wie der globalen Optimierung mittels "Branch and Bound" Algorithmen, der Robotik oder der Computergraphik die Intervallrechnung mit Erfolg eingesetzt. Im Gegensatz zu der vom "Institute of Electrical and Electronics Engineers" IEEE standardisierten Gleitkommaarithmetik, existiert f"ur die Intervallrechnung jedoch keine Norm oder Implementierungsvorschrift. Viele verschiedene Implementierungen und Konzepte wurden über die Jahre entwickelt und auch eingesetzt. Eine einheitliche Semantik wie bei den Gleitkommazahlen ist jedoch nicht gegeben. Um die Unterst"utzung der Intervallarithmetik auf Rechnern zu verbessern, aber auch um die Akzeptanz bei Benutzern weiter zu erhöhen wird aktuell an einem IEEE Standard für Intervallrechnung gearbeitet. Diese Diplomarbeit beschäftigt sich mit den M"oglichkeiten einer effizienten Implementierung der Intervallarithmetik auf Computersystemen mittels der Programmiersprache C++.

  @mastersthesis{nehmeierDA, abstract = {Im Bereich des wissenschaftlichen Rechnens st{\"o}{ss}t die Intervallarithmetik in den letzten Jahren auf immer mehr Akzeptanz. Eigenschaften, wie das sichere Einschlie{ss}en der Ergebnisse bei der Intervallauswertung, machen sie zu einem guten Mittel um mit den Rundungsfehlern der Gleitkommazahlen umzugehen. Auch k{\"o}nnen Berechnungen auf Wertemengen mit garantierten Grenzen durchgef{\"u}hrt werden. Ebenso wird in Anwendungsgebieten wie der globalen Optimierung mittels "Branch and Bound" Algorithmen, der Robotik oder der Computergraphik die Intervallrechnung mit Erfolg eingesetzt. Im Gegensatz zu der vom "Institute of Electrical and Electronics Engineers" IEEE standardisierten Gleitkommaarithmetik, existiert f"ur die Intervallrechnung jedoch keine Norm oder Implementierungsvorschrift. Viele verschiedene Implementierungen und Konzepte wurden {\"u}ber die Jahre entwickelt und auch eingesetzt. Eine einheitliche Semantik wie bei den Gleitkommazahlen ist jedoch nicht gegeben. Um die Unterst"utzung der Intervallarithmetik auf Rechnern zu verbessern, aber auch um die Akzeptanz bei Benutzern weiter zu erh{\"o}hen wird aktuell an einem IEEE Standard f{\"u}r Intervallrechnung gearbeitet. Diese Diplomarbeit besch{\"a}ftigt sich mit den M"oglichkeiten einer effizienten Implementierung der Intervallarithmetik auf Computersystemen mittels der Programmiersprache C++.}, author = {Nehmeier, Marco}, keywords = {Reliability}, school = {Lehrstuhl f{\"u}r Informatik 2, Universit{\"a}t W{\"u}rzburg}, title = {{E}ffiziente {I}ntervallarithmetik mit {C++}}, type = {Diplomarbeit}, year = 2009 }
• 
  A Performance Test Harness For Publish/Subscribe Middleware. K. Sachs; S. Kounev; S. Appel; A. Buchmann; in SIGMETRICS/Performance 2009 International Conference, Seattle, WA, USA, June 15--19, 2009 (2009).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Publish/subscribe is becoming increasingly popular as communication paradigm for loosely-coupled message exchange. It is used as a building block in major new software architectures and technology domains such as Enterprise Service Bus, Enterprise Application Integration, Service-Oriented Architecture and Event-Driven Architecture. The growing adoption of these technologies leads to a strong need for benchmarks and performance evaluation tools in this area. In this demonstration, we present jms2009-PS, a benchmark for publish/subscribe middleware based on the Java Message Service standard interface.

  @inproceedings{SaKoApBu2009-SIGMETRICS-jms2009_PS, abstract = {Publish/subscribe is becoming increasingly popular as communication paradigm for loosely-coupled message exchange. It is used as a building block in major new software architectures and technology domains such as Enterprise Service Bus, Enterprise Application Integration, Service-Oriented Architecture and Event-Driven Architecture. The growing adoption of these technologies leads to a strong need for benchmarks and performance evaluation tools in this area. In this demonstration, we present jms2009-PS, a benchmark for publish/subscribe middleware based on the Java Message Service standard interface.}, author = {Sachs, Kai and Kounev, Samuel and Appel, Stefan and Buchmann, Alejandro}, booktitle = {SIGMETRICS/Performance 2009 International Conference, Seattle, WA, USA, June 15–19, 2009}, keywords = {SPEC}, month = {06}, title = {{A Performance Test Harness For Publish/Subscribe Middleware}}, year = 2009 }
• 
  Performance-Engineering-Research an der Universität Karlsruhe. S. Kounev; (2009, May).
  • [ BibTeX ]
  @misc{talk-Kounev-infoex09, author = {Kounev, Samuel}, howpublished = {Invited Talk, CA infoexchange 2009 Conference, Mannheim, Germany}, keywords = {Talks}, month = {05}, title = {{Performance-Engineering-Research an der Universit\"{a}t Karlsruhe}}, year = 2009 }
• 
  Using WebLogic Diagnostics Framework to Enable Performance Prediction for Java EE Applications. S. Kounev; (2009, September).
  • [ BibTeX ]
  @misc{talk-Kounev-OraclePres09, author = {Kounev, Samuel}, howpublished = {Presentation to Oracle Corp, USA}, keywords = {Talks}, month = {09}, title = {Using WebLogic Diagnostics Framework to Enable Performance Prediction for Java EE Applications}, year = 2009 }
• 
  Benchmarking of Message-Oriented Middleware. K. Sachs; S. Kounev; S. Appel; A. Buchmann; in Proceedings of the 3rd ACM International Conference on Distributed Event-Based Systems (DEBS-2009), Nashville, TN, USA, July 6-9, 2009 (2009).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  In this poster, we provide an overview of our past and current research in the area of Message-Oriented Middleware (MOM) performance benchmarks. Our main research motivation is a) to gain a better understanding of the performance of MOM, b) to show how to use benchmarks for the evaluation of performance aspects and c)to establish performance modeling techniques. For a better understanding, we first introduce the Java Message Service (JMS) standard. Afterwards, we provide an overview of our work on MOM benchmark development, i.e., we present the SPECjms2007 benchmark and the new jms2009-PS, a test harness designed specifically for JMS-based pub/sub. We outline a new case study with jms2009-PS and present first results of our work-in-progress.

  @inproceedings{SaKoApBu2009-DEBS-MOM_Benchmarking, abstract = {In this poster, we provide an overview of our past and current research in the area of Message-Oriented Middleware (MOM) performance benchmarks. Our main research motivation is a) to gain a better understanding of the performance of MOM, b) to show how to use benchmarks for the evaluation of performance aspects and c)to establish performance modeling techniques. For a better understanding, we first introduce the Java Message Service (JMS) standard. Afterwards, we provide an overview of our work on MOM benchmark development, i.e., we present the SPECjms2007 benchmark and the new jms2009-PS, a test harness designed specifically for JMS-based pub/sub. We outline a new case study with jms2009-PS and present first results of our work-in-progress.}, author = {Sachs, Kai and Kounev, Samuel and Appel, Stefan and Buchmann, Alejandro}, booktitle = {Proceedings of the 3rd ACM International Conference on Distributed Event-Based Systems (DEBS-2009), Nashville, TN, {USA}, July 6-9, 2009}, keywords = {t_poster}, month = {07}, publisher = {ACM, New York, NY, USA}, title = {{Benchmarking of Message-Oriented Middleware}}, year = 2009 }
• 
  Improving QoS Guaranties via Adaptive Packet Scheduling. A. Grzech; P. ’S}}wika}tek; P. Rygielski; in Proceedings of the 16th Polish Teletraffic Symposium (PTS 2009) (2009). 53–56.

  Young Author Best Paper Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper the problem of packet scheduling in the node of packet-switched computer communication network is considered. Packet scheduling in the network node is one of the crucial mechanisms essential for delivery of required level of quality of services (QoS). In order to satisfy QoS guaranties for each connection belonging to one of distinguished traffic classes, packet scheduling algorithm must make decisions basing on current state of the scheduling system (e.g. buffer lengths) and actual characteristics of the serviced traffic (e.g.: connections lengths, packet intensities within connections, etc.). Here we propose a new packet scheduling algorithm based on Weighted Round Robin (WRR), which weights are adapted according to changes of system and traffic characteristics. By means of computer simulation, on representative examples, we show, that utilization of additional knowledge in the process of packet scheduling may improve QoS guaranties for serviced traffic.

  @inproceedings{GrSwRy2009-PTS, abstract = {In this paper the problem of packet scheduling in the node of packet-switched computer communication network is considered. Packet scheduling in the network node is one of the crucial mechanisms essential for delivery of required level of quality of services (QoS). In order to satisfy QoS guaranties for each connection belonging to one of distinguished traffic classes, packet scheduling algorithm must make decisions basing on current state of the scheduling system (e.g. buffer lengths) and actual characteristics of the serviced traffic (e.g.: connections lengths, packet intensities within connections, etc.). Here we propose a new packet scheduling algorithm based on Weighted Round Robin (WRR), which weights are adapted according to changes of system and traffic characteristics. By means of computer simulation, on representative examples, we show, that utilization of additional knowledge in the process of packet scheduling may improve QoS guaranties for serviced traffic.}, author = {Grzech, Adam and {\'S}wi\k{a}tek, Pawel and Rygielski, Piotr}, booktitle = {Proceedings of the 16th Polish Teletraffic Symposium (PTS 2009)}, keywords = {Isolation}, month = {09}, note = {<b>Young Author Best Paper Award</b>}, organization = {Technical University of \L{}{\'o}d{\'Z}}, pages = {53–56}, title = {{Improving QoS Guaranties via Adaptive Packet Scheduling}}, year = 2009 }
• 
  Automated Model-based Capacity Planning for Java EE applications. S. Kounev; (2009, December).
  • [ BibTeX ]
  @misc{talk-Kounev-Oracle09, author = {Kounev, Samuel}, howpublished = {Invited Talk, Oracle Corp, USA}, keywords = {Talks}, month = 12, title = {{Automated Model-based Capacity Planning for Java EE applications}}, year = 2009 }
• 
  The Architecture Documentation Maturity Model ADM2. C. Rathfelder; H. Groenda; in Proceedings of the 3rd Workshop MDD, SOA und IT-Management (MSI 2009) (2009). 65–80.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Today, the architectures of software systems are not stable for their whole lifetime but often adapted driven by business needs. Preserving their quality characteristics beyond each of these changes requires deep knowledge of the requirements and the systems themselves. Proper documentation reduces the risk that knowledge is lost and hence is a base for the system's maintenance in the long-run. However, the influence of architectural documentation on the maintainability of software systems is neglected in current quality assessment methods. They are limited to documentation for anticipated change scenarios and do not provide a general assessment approach. In this paper, we propose a maturity model for architecture documentation. It is shaped relative to growing quality preservation maturity and independent of specific technologies or products. It supports the weighting of necessary effort against reducing long-term risks in the maintenance phase. This allows to take product maintainability requirements into account for selecting an appropriate documentation maturity level.

  @inproceedings{rathfelder2009c, abstract = {Today, the architectures of software systems are not stable for their whole lifetime but often adapted driven by business needs. Preserving their quality characteristics beyond each of these changes requires deep knowledge of the requirements and the systems themselves. Proper documentation reduces the risk that knowledge is lost and hence is a base for the system's maintenance in the long-run. However, the influence of architectural documentation on the maintainability of software systems is neglected in current quality assessment methods. They are limited to documentation for anticipated change scenarios and do not provide a general assessment approach. In this paper, we propose a maturity model for architecture documentation. It is shaped relative to growing quality preservation maturity and independent of specific technologies or products. It supports the weighting of necessary effort against reducing long-term risks in the maintenance phase. This allows to take product maintainability requirements into account for selecting an appropriate documentation maturity level.}, address = {Berlin, Germany}, author = {Rathfelder, Christoph and Groenda, Henning}, booktitle = {Proceedings of the 3rd Workshop MDD, SOA und IT-Management (MSI 2009)}, keywords = {t_workshop}, month = 10, pages = {65–80}, publisher = {GiTO-Verlag}, title = {{T}he {A}rchitecture {D}ocumentation {M}aturity {M}odel {ADM2}}, year = 2009 }
• 
  Model-Based Performance Prediction for Event-Driven Systems. C. Rathfelder; S. Kounev; in Proceedings of the Third ACM International Conference on Distributed Event-Based Systems (DEBS 2009) (2009). 33:1–33:2.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The event-driven communication paradigm provides a number of advantages for building loosely coupled distributed systems. However, the loose coupling of components in such systems makes it hard for developers to estimate their behavior and performance under load. Most existing performance prediction techniques for systems using event-driven communication require specialized knowledge to build the necessary prediction models. In this paper, we propose an extension of the Palladio Component Model (PCM) that provides natural support for modeling event-based communication and supports different performance prediction techniques.

  @inproceedings{rathfelder2009, abstract = {The event-driven communication paradigm provides a number of advantages for building loosely coupled distributed systems. However, the loose coupling of components in such systems makes it hard for developers to estimate their behavior and performance under load. Most existing performance prediction techniques for systems using event-driven communication require specialized knowledge to build the necessary prediction models. In this paper, we propose an extension of the Palladio Component Model (PCM) that provides natural support for modeling event-based communication and supports different performance prediction techniques.}, address = {New York, NY, USA}, author = {Rathfelder, Christoph and Kounev, Samuel}, booktitle = {Proceedings of the Third ACM International Conference on Distributed Event-Based Systems (DEBS 2009)}, keywords = {Application_quality_of_service_management}, month = {07}, pages = {33:1–33:2}, publisher = {ACM}, title = {{Model-Based Performance Prediction for Event-Driven Systems}}, year = 2009 }
• 
  Automated Extraction of Palladio Component Models from Running Enterprise Java Applications. F. Brosig; S. Kounev; K. Krogmann; in Proceedings of the 1st International Workshop on Run-time mOdels for Self-managing Systems and Applications (ROSSA 2009). In conjunction with Fourth International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2009), Pisa, Italy, October 19, 2009. (2009).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Nowadays, software systems have to fulfill increasingly stringent requirements for performance and scalability. To ensure that a system meets its performance requirements during operation, the ability to predict its performance under different configurations and workloads is essential. Most performance analysis tools currently used in industry focus on monitoring the current system state. They provide low-level monitoring data without any performance prediction capabilities. For performance prediction, performance models are normally required. However, building predictive performance models manually requires a lot of time and effort. In this paper, we present a method for automated extraction of performance models of Java EE applications, based on monitoring data collected during operation. We extract instances of the Palladio Component Model (PCM) - a performance meta-model targeted at component-based systems. We evaluate the model extraction method in the context of a case study with a real-world enterprise application. Even though the extraction requires some manual intervention, the case study demonstrates that the existing gap between low-level monitoring data and high-level performance models can be closed.

  @inproceedings{BrKoKr2009-ROSSA-Extracting_PCM_JavaEE, abstract = {Nowadays, software systems have to fulfill increasingly stringent requirements for performance and scalability. To ensure that a system meets its performance requirements during operation, the ability to predict its performance under different configurations and workloads is essential. Most performance analysis tools currently used in industry focus on monitoring the current system state. They provide low-level monitoring data without any performance prediction capabilities. For performance prediction, performance models are normally required. However, building predictive performance models manually requires a lot of time and effort. In this paper, we present a method for automated extraction of performance models of Java EE applications, based on monitoring data collected during operation. We extract instances of the Palladio Component Model (PCM) - a performance meta-model targeted at component-based systems. We evaluate the model extraction method in the context of a case study with a real-world enterprise application. Even though the extraction requires some manual intervention, the case study demonstrates that the existing gap between low-level monitoring data and high-level performance models can be closed.}, author = {Brosig, Fabian and Kounev, Samuel and Krogmann, Klaus}, booktitle = {Proceedings of the 1st International Workshop on Run-time mOdels for Self-managing Systems and Applications (ROSSA 2009). In conjunction with Fourth International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2009), Pisa, Italy, October 19, 2009.}, keywords = {t_workshop}, month = 10, publisher = {ACM, New York, NY, USA}, title = {{Automated Extraction of Palladio Component Models from Running Enterprise Java Applications}}, year = 2009 }
• 
  Benchmarking and Performance Modeling of Event-Based Systems. S. Kounev; K. Sachs; in it - Information Technology (2009). 51(5) 262–269.
  • [ Abstract ]
  • [ BibTeX ]
  Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.

  @article{KoSa2009-DEBSPerformance, abstract = {Event-based systems are used increasingly often to build loosely-coupled distributed applications. With their growing popularity and gradual adoption in mission critical areas, the need for novel techniques for benchmarking and performance modeling of event-based systems is increasing. In this article, we provide an overview of the state-of-the-art in this area considering both centralized systems based on message-oriented middleware as well as large-scale distributed publish/subscribe systems. We consider a number of specific techniques for benchmarking and performance modeling, discuss their advantages and disadvantages, and provide references for further information. The techniques we review help to ensure that systems are designed and sized to meet their quality-of-service requirements.}, address = {Munich, Germany}, author = {Kounev, Samuel and Sachs, Kai}, journal = {it - Information Technology}, keywords = {QPN}, month = 10, number = 5, pages = {262–269}, publisher = {Oldenbourg Wissenschaftsverlag}, title = {{Benchmarking and Performance Modeling of Event-Based Systems}}, volume = 51, year = 2009 }
• 
  Using WebLogic Diagnostics Framework to Enable Performance Prediction for Java EE Applications. F. Brosig; S. Kounev; C. Paclat; (2009, September).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Throughout the system life cycle, the ability to predict a software system's performance under different configurations and workloads is highly valuable to ensure that the system meets its performance requirements. During the design phase, performance prediction helps to evaluate different design alternatives. At deployment time, it facilitates system sizing and capacity planning. During operation, predicting the effect of changes in the workload or in the system configuration is beneficial for run-time performance management. The alternative to performance prediction is to deploy the system in an environment reflecting the configuration of interest and conduct experiments measuring the system performance under the respective workloads. Such experiments, however, are normally very expensive and time-consuming and therefore often considered not to be economically viable. To enable performance prediction we need an abstraction of the real system that incorporates performance-relevant data, i.e., a performance model. Based on such a model, performance analysis can be carried out. Unfortunately, building predictive performance models manually requires a lot of time and effort. The model must be designed to reflect the abstract system structure and capture its performance-relevant aspects. In addition, model parameters like resource demands or system configuration parameters have to be determined. Given the costs of building performance models, techniques for automatic extraction of models based on observation of the system at run-time are highly desirable. During system development, such models can be exploited to evaluate the performance of system prototypes. During operation, an automatically extracted performance model can be applied for efficient and performance-aware resource management. For example, if one observes an increased user workload and assumes a steady workload growth rate, performance predictions help to determine when the system would reach its saturation point. This way, system operators can react to the changing workload before the system has failed to meet its performance objectives thus avoiding a violation of service level agreements (SLAs). Current performance analysis tools used in industry mostly focus on profiling and monitoring transaction response times and resource consumption. The tools often provide large amounts of low level data while important information needed for building performance models is missing, e.g., the resource demands of individual components. In this article, we present a method for automated extraction of performance models for Java EE applications during operation. We implemented the method in a tool prototype and evaluated its effectiveness in the context of a case study with an early prototype of the SPECjEnterprise2009 benchmark application which in the following we will refer to as SPECjEnterprise2009_pre. (SPECjEnterprise2009 is the successor benchmark of the SPECjAppServer2004 benchmark developed by the Standard Performance Evaluation Corp. [SPEC]; SPECjEnterprise is a trademark of SPEC. The SPECjEnterprise2009 results or findings in this publication have not been reviewed or accepted by SPEC, therefore no comparison nor performance inference can be made against any published SPEC result.) The target Java EE platform we consider is Oracle WebLogic Server (WLS). The extraction is based on monitoring data that is collected during operation using the WebLogic Diagnostics Framework (WLDF). As a performance model, we selected the Palladio Component Model (PCM). PCM is a sophisticated performance modeling framework with mature tool support. In contrast to low level mathematical models like, e.g., queueing networks, PCM is a high-level UML-like design-oriented model that captures the performance-relevant aspects of the system architecture. This makes PCM models easy to understand and use by software developers. We begin by providing some background on the technologies we use, focusing on the WLDF monitoring framework and the PCM models. We then describe the model extraction method in more detail. Finally, we present the case study we conducted and conclude with a summary.

  @misc{BrKoPa2009-OTN-WLDF2PCM, abstract = {Throughout the system life cycle, the ability to predict a software system's performance under different configurations and workloads is highly valuable to ensure that the system meets its performance requirements. During the design phase, performance prediction helps to evaluate different design alternatives. At deployment time, it facilitates system sizing and capacity planning. During operation, predicting the effect of changes in the workload or in the system configuration is beneficial for run-time performance management. The alternative to performance prediction is to deploy the system in an environment reflecting the configuration of interest and conduct experiments measuring the system performance under the respective workloads. Such experiments, however, are normally very expensive and time-consuming and therefore often considered not to be economically viable. To enable performance prediction we need an abstraction of the real system that incorporates performance-relevant data, i.e., a performance model. Based on such a model, performance analysis can be carried out. Unfortunately, building predictive performance models manually requires a lot of time and effort. The model must be designed to reflect the abstract system structure and capture its performance-relevant aspects. In addition, model parameters like resource demands or system configuration parameters have to be determined. Given the costs of building performance models, techniques for automatic extraction of models based on observation of the system at run-time are highly desirable. During system development, such models can be exploited to evaluate the performance of system prototypes. During operation, an automatically extracted performance model can be applied for efficient and performance-aware resource management. For example, if one observes an increased user workload and assumes a steady workload growth rate, performance predictions help to determine when the system would reach its saturation point. This way, system operators can react to the changing workload before the system has failed to meet its performance objectives thus avoiding a violation of service level agreements (SLAs). Current performance analysis tools used in industry mostly focus on profiling and monitoring transaction response times and resource consumption. The tools often provide large amounts of low level data while important information needed for building performance models is missing, e.g., the resource demands of individual components. In this article, we present a method for automated extraction of performance models for Java EE applications during operation. We implemented the method in a tool prototype and evaluated its effectiveness in the context of a case study with an early prototype of the SPECjEnterprise2009 benchmark application which in the following we will refer to as SPECjEnterprise2009_pre. (SPECjEnterprise2009 is the successor benchmark of the SPECjAppServer2004 benchmark developed by the Standard Performance Evaluation Corp. [SPEC]; SPECjEnterprise is a trademark of SPEC. The SPECjEnterprise2009 results or findings in this publication have not been reviewed or accepted by SPEC, therefore no comparison nor performance inference can be made against any published SPEC result.) The target Java EE platform we consider is Oracle WebLogic Server (WLS). The extraction is based on monitoring data that is collected during operation using the WebLogic Diagnostics Framework (WLDF). As a performance model, we selected the Palladio Component Model (PCM). PCM is a sophisticated performance modeling framework with mature tool support. In contrast to low level mathematical models like, e.g., queueing networks, PCM is a high-level UML-like design-oriented model that captures the performance-relevant aspects of the system architecture. This makes PCM models easy to understand and use by software developers. We begin by providing some background on the technologies we use, focusing on the WLDF monitoring framework and the PCM models. We then describe the model extraction method in more detail. Finally, we present the case study we conducted and conclude with a summary.}, author = {Brosig, Fabian and Kounev, Samuel and Paclat, Charles}, howpublished = {Oracle Technology Network (OTN) Article}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = {09}, title = {{Using WebLogic Diagnostics Framework to Enable Performance Prediction for Java EE Applications}}, year = 2009 }
• 
  Twostage Packet Scheduling in the Network Node. A. Grzech; P. ’S}}wika}tek; P. Rygielski; in Proceedings of Twentieth International Conference on Systems Engineering, (ICSE 2009), K. J. Burnham, O. C. L. Haas (eds.) (2009). 181–184.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper the problem of two-stage packet scheduling on parallel processors is considered. It is assumed, that each processor schedules packets according to Weighted Round Robin (WRR) rule. In order to deliver required level of the quality of service (QoS) weights of local WRR algorithms are adapted such that QoS requirements are met for all distinguished traffic classes. Adaptation of WRR weights relies on the adaptation through identification (Bubnicki, 2005) methodology with the diagonal recurrent neural network (DRNN) applied as the model of QoS parameters. Simulation study of the performance of proposed scheduling methodology shows, that it can deliver better QoS guaranties and higher systems throughput than commonly used reference algorithms.

  @inproceedings{GrSwRy2009-ICSE, abstract = {In this paper the problem of two-stage packet scheduling on parallel processors is considered. It is assumed, that each processor schedules packets according to Weighted Round Robin (WRR) rule. In order to deliver required level of the quality of service (QoS) weights of local WRR algorithms are adapted such that QoS requirements are met for all distinguished traffic classes. Adaptation of WRR weights relies on the adaptation through identification (Bubnicki, 2005) methodology with the diagonal recurrent neural network (DRNN) applied as the model of QoS parameters. Simulation study of the performance of proposed scheduling methodology shows, that it can deliver better QoS guaranties and higher systems throughput than commonly used reference algorithms.}, author = {Grzech, Adam and {\'S}wi\k{a}tek, Pawel and Rygielski, Piotr}, booktitle = {Proceedings of Twentieth International Conference on Systems Engineering, (ICSE 2009)}, editor = {Burnham, Keith J. and Haas, Olivier C. L.}, keywords = {descartes}, month = {09}, pages = {181–184}, publisher = {Coventry University Press}, title = {{Twostage Packet Scheduling in the Network Node}}, year = 2009 }
• 
  Modeling Event-Driven Service-Oriented Systems using the Palladio Component Model. C. Rathfelder; S. Kounev; in Proceedings of the 1st International Workshop on the Quality of Service-Oriented Software Systems (QUASOSS 2009) (2009). 33–38.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The use of event-based communication within a Service-Oriented Architecture promises several benefits including more loosely-coupled services and better scalability. However, the loose coupling of services makes it difficult for system developers to estimate the behavior and performance of systems composed of multiple services. Most existing performance prediction techniques for systems using event-based communication require specialized knowledge to build the necessary prediction models. Furthermore, general purpose design-oriented performance models for component-based systems provide limited support for modeling event-based communication. In this paper, we propose an extension of the Palladio Component Model (PCM) that provides natural support for modeling event-based communication. We show how this extension can be exploited to model event-driven service-oriented systems with the aim to evaluate their performance and scalability.

  @inproceedings{rathfelder2009b, abstract = {The use of event-based communication within a Service-Oriented Architecture promises several benefits including more loosely-coupled services and better scalability. However, the loose coupling of services makes it difficult for system developers to estimate the behavior and performance of systems composed of multiple services. Most existing performance prediction techniques for systems using event-based communication require specialized knowledge to build the necessary prediction models. Furthermore, general purpose design-oriented performance models for component-based systems provide limited support for modeling event-based communication. In this paper, we propose an extension of the Palladio Component Model (PCM) that provides natural support for modeling event-based communication. We show how this extension can be exploited to model event-driven service-oriented systems with the aim to evaluate their performance and scalability.}, address = {New York, USA}, author = {Rathfelder, Christoph and Kounev, Samuel}, booktitle = {Proceedings of the 1st International Workshop on the Quality of Service-Oriented Software Systems (QUASOSS 2009)}, keywords = {t_workshop}, month = {08}, pages = {33–38}, publisher = {ACM}, title = {{M}odeling {E}vent-{D}riven {S}ervice-{O}riented {S}ystems using the {P}alladio {C}omponent {M}odel}, year = 2009 }
• 
  Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark. K. Sachs; S. Kounev; J. Bacon; A. Buchmann; in Performance Evaluation (2009). 66(8) 410–434.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.

  @article{SaKoBaBu2008-PERFEVAL-SPECjms2007, abstract = {Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. Novel messaging applications, however, pose some serious performance and scalability challenges. In this paper, we present a methodology for performance evaluation of MOM platforms using the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. SPECjms2007 is based on a novel application in the supply chain management domain designed to stress MOM infrastructures in a manner representative of real-world applications. In addition to providing a standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to tailor the workload to their requirements. The contributions of this paper are: i) we present a detailed workload characterization of SPECjms2007 with the goal to help users understand the internal components of the workload and the way they are scaled, ii) we show how the workload can be customized to exercise and evaluate selected aspects of MOM performance, iii) we present a case study of a leading JMS platform, the BEA WebLogic server, conducting an in-depth performance analysis of the platform under a number of different workload and configuration scenarios. The methodology we propose is the first one that uses an industry-standard benchmark providing both a representative workload as well as the ability to customize it to evaluate the features of MOM platforms selectively.}, address = {Amsterdam, The Netherlands}, author = {Sachs, Kai and Kounev, Samuel and Bacon, Jean and Buchmann, Alejandro}, journal = {Performance Evaluation}, keywords = {SPEC}, month = {08}, number = 8, pages = {410–434}, publisher = {Elsevier Science Publishers B. V.}, title = {{Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark}}, volume = 66, year = 2009 }

• 
  SPECjms2007: A Novel Benchmark and Performance Analysis Framework for Message-Oriented Middleware. S. Kounev; K. Sachs; (2008, March).
  • [ BibTeX ]
  • [ URL ]
  @misc{kounev2008b, author = {Kounev, Samuel and Sachs, Kai}, howpublished = {DEV2DEV Article, O'Reilly Publishing Group}, keywords = {SPEC}, month = {03}, title = {{SPECjms2007: A Novel Benchmark and Performance Analysis Framework for Message-Oriented Middleware}}, year = 2008 }
• 
  QPME (Queueing Petri net Modeling Environment) Homepage. S. Kounev; (2008).
  • [ BibTeX ]
  • [ URL ]
  @misc{kounev2008a, author = {Kounev, Samuel}, howpublished = {{Online, \url{http://descartes.ipd.kit.edu/projects/QPME}}}, keywords = {QPN}, title = {{QPME (Queueing Petri net Modeling Environment) Homepage}}, year = 2008 }
• 
  Kaffeekunde - SPECjms misst Message-oriented Middleware. S. Kounev; K. Sachs; (2008).
  • [ BibTeX ]
  • [ URL ]
  @misc{sachs2008a, author = {Kounev, Samuel and Sachs, Kai}, howpublished = {iX Magazin, Heft 02/2008, Heise Zeitschriften Verlag}, keywords = {SPEC}, title = {{Kaffeekunde - SPECjms misst Message-oriented Middleware}}, year = 2008 }
• 
  Software-Industrialisierung F. Brosch; T. Goldschmidt; H. Groenda; L. Kapova; K. Krogmann; M. Kuperberg; A. Martens; C. Rathfelder; R. Reussner; J. Stammel; (2008). (1432nd-7864th ed., Vol. 8)
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  Die Industrialisierung der Software-Entwicklung ist ein zurzeit sehr stark diskutiertes Thema. Es geht dabei vor allem um die Efizienzsteigerung durch die Steigerung des Standardisierungsgrades, des Automatisierungsgrades sowie eine Erhöhung der Arbeitsteilung. Dies wirkt sich einerseits auf die den Software- Systemen zu Grunde liegenden Architekturen aber auch auf die Entwicklungsprozesse aus. So sind service-orientierte Architekturen ein Beispiel für eine gesteigerte Standardisierung innerhalb von Software-Systemen. Es ist zu berücksichtigen, dass sich die Software-Branche von den klassischen produzierenden Industriezweigen dadurch unterscheidet, dass Software ein immaterielles Produkt ist und so ohne hohe Produktionskosten beliebig oft vervielfältigt werden kann. Trotzdem lassen sich viele Erkenntnisse aus den klassischen Industriezweigen auf die Software-Technik übertragen. Die Inhalte dieses Berichts stammen hauptsächlich aus dem Seminar "Software- Industrialisierung, welches sich mit der Professionalisierung der Software- Entwicklung und des Software-Entwurfs beschäftigte. Während die klassische Software-Entwicklung wenig strukturiert ist und weder im Bezug auf Reproduzierbarkeit oder Qualitätssicherung erhöhten Anforderungen genügt, befindet sich die Software-Entwicklung im Rahmen der Industrialisierung in einem Wandel. Dazu zählen arbeitsteiliges Arbeiten, die Einführung von Entwicklungsprozessen mit vorhersagbaren Eigenschaften (Kosten, Zeitbedarf, ...) und in der Folge die Erstellung von Produkten mit garantierbaren Eigenschaften. Das Themenspektrum des Seminars umfasste dabei unter anderem: * Software-Architekturen * Komponentenbasierte Software-Entwicklung * Modellgetriebene Entwicklung * Berücksichtigung von Qualitätseigenschaften in Entwicklungsprozessen Das Seminar wurde wie eine wissenschaftliche Konferenz organisiert: Die Einreichungen wurden in einem zweistufigen Peer-Review-Verfahren begutachtet. In der ersten Stufe wurde eine Begutachtung der studentischen Arbeiten durch Kommilitonen durchgeführt, in der zweiten Stufe eine Begutachtung durch die Betreuer. In verschiedenen Sessions wurden die Artikel an zwei Konferenztagen präsentiert. Der beste Beitrag wurde durch einen Best Paper Award ausgezeichnet. Dieser ging an Benjamin Klatt für seine Arbeit Software Extension Mechanisms, dem hiermit noch einmal herzlich zu dieser herausragenden Leistung gratuliert wird. Ergänzend zu den Vorträgen der Seminarteilnehmer wurde ein eingeladener Vortrag gehalten. Herr Florian Kaltner und Herr Tobias Pohl vom IBM-Entwicklungslabor gaben dabei dankenswerterweise in ihrem Vortrag Einblicke in die Entwicklung von Plugins für Eclipse sowie in die Build-Umgebung der Firmware für die zSeries Mainframe-Server.

  @techreport{indust2008, abstract = {Die Industrialisierung der Software-Entwicklung ist ein zurzeit sehr stark diskutiertes Thema. Es geht dabei vor allem um die Efizienzsteigerung durch die Steigerung des Standardisierungsgrades, des Automatisierungsgrades sowie eine Erh{\"o}hung der Arbeitsteilung. Dies wirkt sich einerseits auf die den Software- Systemen zu Grunde liegenden Architekturen aber auch auf die Entwicklungsprozesse aus. So sind service-orientierte Architekturen ein Beispiel f{\"u}r eine gesteigerte Standardisierung innerhalb von Software-Systemen. Es ist zu ber{\"u}cksichtigen, dass sich die Software-Branche von den klassischen produzierenden Industriezweigen dadurch unterscheidet, dass Software ein immaterielles Produkt ist und so ohne hohe Produktionskosten beliebig oft vervielf{\"a}ltigt werden kann. Trotzdem lassen sich viele Erkenntnisse aus den klassischen Industriezweigen auf die Software-Technik {\"u}bertragen. Die Inhalte dieses Berichts stammen haupts{\"a}chlich aus dem Seminar "Software- Industrialisierung, welches sich mit der Professionalisierung der Software- Entwicklung und des Software-Entwurfs besch{\"a}ftigte. W{\"a}hrend die klassische Software-Entwicklung wenig strukturiert ist und weder im Bezug auf Reproduzierbarkeit oder Qualit{\"a}tssicherung erh{\"o}hten Anforderungen gen{\"u}gt, befindet sich die Software-Entwicklung im Rahmen der Industrialisierung in einem Wandel. Dazu z{\"a}hlen arbeitsteiliges Arbeiten, die Einf{\"u}hrung von Entwicklungsprozessen mit vorhersagbaren Eigenschaften (Kosten, Zeitbedarf, ...) und in der Folge die Erstellung von Produkten mit garantierbaren Eigenschaften. Das Themenspektrum des Seminars umfasste dabei unter anderem: * Software-Architekturen * Komponentenbasierte Software-Entwicklung * Modellgetriebene Entwicklung * Ber{\"u}cksichtigung von Qualit{\"a}tseigenschaften in Entwicklungsprozessen Das Seminar wurde wie eine wissenschaftliche Konferenz organisiert: Die Einreichungen wurden in einem zweistufigen Peer-Review-Verfahren begutachtet. In der ersten Stufe wurde eine Begutachtung der studentischen Arbeiten durch Kommilitonen durchgef{\"u}hrt, in der zweiten Stufe eine Begutachtung durch die Betreuer. In verschiedenen Sessions wurden die Artikel an zwei Konferenztagen pr{\"a}sentiert. Der beste Beitrag wurde durch einen Best Paper Award ausgezeichnet. Dieser ging an Benjamin Klatt f{\"u}r seine Arbeit Software Extension Mechanisms, dem hiermit noch einmal herzlich zu dieser herausragenden Leistung gratuliert wird. Erg{\"a}nzend zu den Vortr{\"a}gen der Seminarteilnehmer wurde ein eingeladener Vortrag gehalten. Herr Florian Kaltner und Herr Tobias Pohl vom IBM-Entwicklungslabor gaben dabei dankenswerterweise in ihrem Vortrag Einblicke in die Entwicklung von Plugins f{\"u}r Eclipse sowie in die Build-Umgebung der Firmware f{\"u}r die zSeries Mainframe-Server.}, address = {Karlsruhe}, author = {Brosch, Franz and Goldschmidt, Thomas and Groenda, Henning and Kapova, Lucia and Krogmann, Klaus and Kuperberg, Michael and Martens, Anne and Rathfelder, Christoph and Reussner, Ralf and Stammel, Johannes}, edition = {ISSN 1432-7864}, institution = {Universit{"a}t Karlsruhe, Fakult{"a}t f{"u}r Informatik, Institut f{"u}r Programmstrukturen und Datenorganisation}, keywords = {se2}, title = {Software-Industrialisierung}, type = {Interner Bericht}, volume = 8, year = 2008 }
• 
  Software Zertifizierung. (4/2008), J. Blomer; F. Brosig; A. Kreidler; J. K"uttel; A. Kuwertz; G. Liebel; D. Popovic; M. St"ubs; A. M. Turek; C. Vogel; T. Weinstein; T. Wurth; (T. Goldschmidt; H. Groenda; K. Krogmann; M. Kuperberg; A. Martens; C. Rathfelder; R. Reussner; J. Stammel, eds.) (2008).
  • [ Abstract ]
  • [ BibTeX ]
  Systematische Qualitätssicherung gewinnt im Rahmen des globalenWettbewerbs auch in der Software-Entwicklungsbranche zunehmend an Bedeutung. Vor allem auf dem Weg zur Software-Industrialisierung bzw. zu einer ingenieurmässigen Software-Entwicklung ist eine durchgängige Qualitätssicherung unabdingbar. Zertifizierungen bieten hierbei die Möglichkeit, die Einhaltung bestimmter Standards und Kriterien durch unabhängige Dritte überprüfen und bescheinigen zu lassen, um die Qualität eines Produktes oder Entwicklungsprozesses zu belegen. Zertifizierungen können sich sowohl auf Produkte und Prozesse als auch auf die Ausbildung und das Wissen von Einzelpersonen beziehen. Da Zertifikate durch unabhängige Prüfinstanzen ausgestellt werden, wird Zertifikaten und deren überprüfbaren Aussagen im Allgemeinen ein deutlich höheres Vertrauen entgegengebracht als Qualitätsversprechen von Software-Herstellern selbst. Unternehmen, die ihre Prozesse beispielsweise nach CMMI zertifizieren lassen, können damit ihre Fähigkeiten unter Beweis stellen, Projekte erfolgreich und mit vorhersagbarer Qualität abschliessen zu können. Neben dem Nachweis entsprechender Zertifikate als Diversifikationsmerkmal gegenüber Mitbewerbern können Zertifikate über die Einhaltung von Standards auch durch den Gesetzgeber vorgeschrieben werden. Ein Beispiel hierfür sind Zertifikate aus Hochsicherheitsbereichen wie Atomkraftwerken. Das Seminar wurde wie eine wissenschaftliche Konferenz organisiert: Die Einreichungen wurden in einem zweistufigen Peer-Review-Verfahren begutachtet. In der ersten Stufe wurde eine Begutachtung der studentischen Arbeiten durch Kommilitonen durchgeführt, in der zweiten Stufe eine Begutachtung durch die Betreuer. In verschiedenen Sessions wurden die Artikel an zwei Konferenztagen präsentiert. Die besten Beiträge wurden durch best paper awards ausgezeichnet. Diese gingen an Fabian Brosig für seine Arbeit Cost Benefit Analysis Method (CBAM), an Jakob Blomer für die Arbeit Zertifizierung von Softwarebenchmarks und an Grischa Liebel für die Arbeit SWT - Das Standard Widget Toolkit, denen hiermit noch einmal herzlich zu dieser herausragenden Leistung gratuliert wird. Ergänzend zu den Vorträgen der Seminarteilnehmer wurde ein eingeladener Vortrag gehalten. Herr Dr. Dirk Feuerhelm von der 1&1 Internet AG gab dabei dankenswerterweise in seinem Vortrag mit dem Thema Softskills -- Ist das objektorientiert oder modellgetrieben? einen Einblick in die Aufgaben als Leiter der Software-Entwicklung

  @techreport{groenda2008c, abstract = {Systematische Qualit{\"a}tssicherung gewinnt im Rahmen des globalenWettbewerbs auch in der Software-Entwicklungsbranche zunehmend an Bedeutung. Vor allem auf dem Weg zur Software-Industrialisierung bzw. zu einer ingenieurm{\"a}ss{}igen Software-Entwicklung ist eine durchg{\"a}ngige Qualit{\"a}tssicherung unabdingbar. Zertifizierungen bieten hierbei die M{\"o}glichkeit, die Einhaltung bestimmter Standards und Kriterien durch unabh{\"a}ngige Dritte {\"u}berpr{\"u}fen und bescheinigen zu lassen, um die Qualit{\"a}t eines Produktes oder Entwicklungsprozesses zu belegen. Zertifizierungen k{\"o}nnen sich sowohl auf Produkte und Prozesse als auch auf die Ausbildung und das Wissen von Einzelpersonen beziehen. Da Zertifikate durch unabh{\"a}ngige Pr{\"u}finstanzen ausgestellt werden, wird Zertifikaten und deren {\"u}berpr{\"u}fbaren Aussagen im Allgemeinen ein deutlich h{\"o}heres Vertrauen entgegengebracht als Qualit{\"a}tsversprechen von Software-Herstellern selbst. Unternehmen, die ihre Prozesse beispielsweise nach CMMI zertifizieren lassen, k{\"o}nnen damit ihre F{\"a}higkeiten unter Beweis stellen, Projekte erfolgreich und mit vorhersagbarer Qualit{\"a}t abschlie{ss}en zu k{\"o}nnen. Neben dem Nachweis entsprechender Zertifikate als Diversifikationsmerkmal gegen{\"u}ber Mitbewerbern k{\"o}nnen Zertifikate {\"u}ber die Einhaltung von Standards auch durch den Gesetzgeber vorgeschrieben werden. Ein Beispiel hierf{\"u}r sind Zertifikate aus Hochsicherheitsbereichen wie Atomkraftwerken. Das Seminar wurde wie eine wissenschaftliche Konferenz organisiert: Die Einreichungen wurden in einem zweistufigen Peer-Review-Verfahren begutachtet. In der ersten Stufe wurde eine Begutachtung der studentischen Arbeiten durch Kommilitonen durchgef{\"u}hrt, in der zweiten Stufe eine Begutachtung durch die Betreuer. In verschiedenen Sessions wurden die Artikel an zwei Konferenztagen pr{\"a}sentiert. Die besten Beitr{\"a}ge wurden durch best paper awards ausgezeichnet. Diese gingen an Fabian Brosig f{\"u}r seine Arbeit Cost Benefit Analysis Method (CBAM), an Jakob Blomer f{\"u}r die Arbeit Zertifizierung von Softwarebenchmarks und an Grischa Liebel f{\"u}r die Arbeit SWT - Das Standard Widget Toolkit, denen hiermit noch einmal herzlich zu dieser herausragenden Leistung gratuliert wird. Erg{\"a}nzend zu den Vortr{\"a}gen der Seminarteilnehmer wurde ein eingeladener Vortrag gehalten. Herr Dr. Dirk Feuerhelm von der 1&1 Internet AG gab dabei dankenswerterweise in seinem Vortrag mit dem Thema Softskills – Ist das objektorientiert oder modellgetrieben? einen Einblick in die Aufgaben als Leiter der Software-Entwicklung}, author = {Blomer, Jakob and Brosig, Fabian and Kreidler, Andreas and K{"u}ttel, Jens and Kuwertz, Achim and Liebel, Grischa and Popovic, Daniel and St{"u}bs, Michael and Turek, Alexander M. and Vogel, Christian and Weinstein, Thomas and Wurth, Thomas}, editor = {Goldschmidt, Thomas and Groenda, Henning and Krogmann, Klaus and Kuperberg, Michael and Martens, Anne and Rathfelder, Christoph and Reussner, Ralf and Stammel, Johannes}, institution = {Universit{\"a}t Karlsruhe, Fakult{\"a}t f{\"u}r Informatik}, keywords = {se2}, number = {4/2008}, title = {{S}oftware {Z}ertifizierung}, year = 2008 }
• 
  Performance Prediction using QPN Models: From Capacity Planning to Online Performance Management. S. Kounev; (2008, January).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Charles08, author = {Kounev, Samuel}, howpublished = {Invited Talk, Department of Software Engineering, Faculty of Mathematics and Physics, Charles University, Prague, Czech Republic}, keywords = {Talks}, month = {01}, title = {{Performance Prediction using QPN Models: From Capacity Planning to Online Performance Management}}, year = 2008 }
• 
  Autonomic QoS control in enterprise Grid environments using online simulation. S. Kounev; (2008, February).
  • [ BibTeX ]
  @misc{talk-Kounev-NewcastleTyne08, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), School of Computing Science, Newcastle University, Newcastle upon Tyne, UK}, keywords = {Talks}, month = {02}, title = {{Autonomic QoS control in enterprise Grid environments using online simulation}}, year = 2008 }
• 
  Towards Autonomic QoS Management in Event-based and Service-Oriented Environments. S. Kounev; (2008, March).
  • [ BibTeX ]
  @misc{talk-Kounev-Cambridge08, author = {Kounev, Samuel}, howpublished = {University of Cambridge, Computer Laboratory, Cambridge, UK}, keywords = {Talks}, month = {03}, title = {{Towards Autonomic QoS Management in Event-based and Service-Oriented Environments}}, year = 2008 }
• 
  A Methodology for Performance Modeling of Distributed Event-Based Systems. S. Kounev; K. Sachs; J. Bacon; A. Buchmann; in Proceedings of the 11th IEEE International Symposium on Object Oriented Real-Time Distributed Computing (ISORC 2008), Orlando, Florida, USA, May 5-7, 2008 (2008). 13–22.

  Acceptance Rate (Full Paper): 30%, Best-Paper-Award-Nomination
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Distributed event-based systems (DEBS) are gaining increasing attention in new application areas such as transport information monitoring, event-driven supply-chain management and ubiquitous sensor-rich environments. However, as DEBS increasingly enter the enterprise and commercial domains, performance and quality of service issues are becoming a major concern. While numerous approaches to performance modeling and evaluation of conventional request/reply-based distributed systems are available in the literature, no general approach exists for DEBS. This paper is the first to provide a comprehensive methodology for workload characterization and performance modeling of DEBS. A workload model of a generic DEBS is developed and operational analysis techniques are used to characterize the system traffic and derive an approximation for the mean event delivery latency. Following this, a modeling technique is presented that can be used for accurate performance prediction. The paper is concluded with a case study of a real life system demonstrating the effectiveness and practicality of the proposed approach.

  @inproceedings{KoSaBaBu2008-ISORC-ModelingDEBS, abstract = {Distributed event-based systems (DEBS) are gaining increasing attention in new application areas such as transport information monitoring, event-driven supply-chain management and ubiquitous sensor-rich environments. However, as DEBS increasingly enter the enterprise and commercial domains, performance and quality of service issues are becoming a major concern. While numerous approaches to performance modeling and evaluation of conventional request/reply-based distributed systems are available in the literature, no general approach exists for DEBS. This paper is the first to provide a comprehensive methodology for workload characterization and performance modeling of DEBS. A workload model of a generic DEBS is developed and operational analysis techniques are used to characterize the system traffic and derive an approximation for the mean event delivery latency. Following this, a modeling technique is presented that can be used for accurate performance prediction. The paper is concluded with a case study of a real life system demonstrating the effectiveness and practicality of the proposed approach.}, address = {Washington, DC, USA}, author = {Kounev, Samuel and Sachs, Kai and Bacon, Jean and Buchmann, Alejandro}, booktitle = {Proceedings of the 11th IEEE International Symposium on Object Oriented Real-Time Distributed Computing (ISORC 2008), Orlando, Florida, USA, May 5-7, 2008}, keywords = {QPN}, note = {Acceptance Rate (Full Paper): 30\%, <b>Best-Paper-Award-Nomination</b>}, pages = {13–22}, publisher = {IEEE Computer Society}, title = {{A Methodology for Performance Modeling of Distributed Event-Based Systems}}, year = 2008 }
• 
  Manageability Design for an Autonomic Management of Semi-Dynamic Web Service Compositions. C. Momm; C. Rathfelder; I. P. Hallerbach; S. Abeck; in Proceedings of the Network Operations and Management Symposium (NOMS 2008) (2008). 839–842.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Web service compositions (WSC), as part of a service- oriented architecture (SOA), have to be managed to ensure compliance with guaranteed service levels. In this context, a high degree of automation is desired, which can be achieved by applying autonomic computing concepts. This paper particularly focuses the autonomic management of semi-dynamic compositions. Here, for each included service several variants are available that differ with regard to the service level they offer. Given this scenario, we first show how to instrument WSC in order to allow a controlling of the service level through switching the employed service variant. Second, we show how the desired self-manageability can be designed and implemented by means of a WSC manageability infrastructure. The presented approach is based on widely accepted methodologies and standards from the area of application and web service management, in particular the WBEM standards.

  @inproceedings{momm2008a, abstract = {Web service compositions (WSC), as part of a service- oriented architecture (SOA), have to be managed to ensure compliance with guaranteed service levels. In this context, a high degree of automation is desired, which can be achieved by applying autonomic computing concepts. This paper particularly focuses the autonomic management of semi-dynamic compositions. Here, for each included service several variants are available that differ with regard to the service level they offer. Given this scenario, we first show how to instrument WSC in order to allow a controlling of the service level through switching the employed service variant. Second, we show how the desired self-manageability can be designed and implemented by means of a WSC manageability infrastructure. The presented approach is based on widely accepted methodologies and standards from the area of application and web service management, in particular the WBEM standards.}, author = {Momm, Christof and Rathfelder, Christoph and Hallerbach, Ignacio P{\'e}rez and Abeck, Sebastian}, booktitle = {Proceedings of the Network Operations and Management Symposium (NOMS 2008)}, keywords = {se2}, month = {04}, pages = {839–842}, publisher = {IEEE}, title = {{M}anageability {D}esign for an {A}utonomic {M}anagement of {S}emi-{D}ynamic {W}eb {S}ervice {C}ompositions}, year = 2008 }
• 
  Descartes Projekt - Autonomes Performanz-und Ressourcen-Management in dynamischen, dienstorientierten Umgebungen. S. Kounev; (2008, September).
  • [ BibTeX ]
  @misc{talk-Kounev-DFG08, author = {Kounev, Samuel}, howpublished = {Invited Talk, DFG Fachforum Informatik, Deutsche Forschungsgemeinschaft (German Research Foundation), Bonn, Germany}, keywords = {Talks}, month = {09}, title = {{Descartes Projekt - Autonomes Performanz-und Ressourcen-Management in dynamischen, dienstorientierten Umgebungen}}, year = 2008 }
• 
  Software Industrialization and Architecture Certification. C. Rathfelder; H. Groenda; R. Reussner; in Industrialisierung des Software-Managements: Fachtagung des GI-Fachausschusses Management der Anwendungsentwicklung und -Wartung im Fachbereich Wirtschaftsinformatik (WI-MAW), G. Herzwurm, M. Mikusz (eds.) (2008). (Vol. 139) 169–180.
  • [ BibTeX ]
  @incollection{rathfelder2008b, author = {Rathfelder, Christoph and Groenda, Henning and Reussner, Ralf}, booktitle = {Industrialisierung des Software-Managements: Fachtagung des GI-Fachausschusses Management der Anwendungsentwicklung und -Wartung im Fachbereich Wirtschaftsinformatik (WI-MAW)}, editor = {Herzwurm, Georg and Mikusz, Martin}, keywords = {se2}, month = 11, pages = {169–180}, series = {Lecture Notes in Informatics (LNI)}, title = {{S}oftware {I}ndustrialization and {A}rchitecture {C}ertification}, volume = 139, year = 2008 }
• 
  Towards an Architecture Maintainability Maturity Model ({AM}3). C. Rathfelder; H. Groenda; in Softwaretechnik-Trends (2008). 28(4) 3–7.
  • [ BibTeX ]
  • [ Download ]
  @article{rathfelder2008c, address = {Bonn, Germany}, author = {Rathfelder, Christoph and Groenda, Henning}, journal = {Softwaretechnik-Trends}, keywords = {se2}, month = 11, number = 4, pages = {3–7}, publisher = {GI (Gesellschaft fuer Informatik)}, title = {{T}owards an {A}rchitecture {M}aintainability {M}aturity {M}odel ({AM}3)}, volume = 28, year = 2008 }
• 
  Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28 S. Kounev; in Lecture Notes in Computer Science (LNCS), (S. Kounev; I. Gorton; K. Sachs, eds.) (2008). (Vol. 5119) Springer, Heidelberg, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.

  @proceedings{kounev2008c, abstract = {This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.}, address = {Heidelberg, Germany}, author = {Kounev, Samuel}, editor = {Kounev, Samuel and Gorton, Ian and Sachs, Kai}, keywords = {Power}, month = {06}, publisher = {Springer}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28}}, volume = 5119, year = 2008 }
• 
  Model-based Management of Web Service Compositions in Service-Oriented Architectures. C. Momm; C. Rathfelder; in MDD, SOA und IT-Management (MSI 2008), U. Steffens, J. S. Addicks, N. Streekmann (eds.) (2008). 25–40.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Web service compositions (WSC), as part of a service-oriented architecture (SOA), have to be managed to ensure compliance with guaranteed service levels. In this context, a high degree of automation is desired, which can be achieved by applying autonomic computing concepts. This paper particularly focuses the autonomic management of semi-dynamic compositions. Here, for each included service several variants are available that differ with regard to the service level they offer. Given this scenario, we first show how to instrument WSC in order to allow a controlling of the service level through switching the employed service variant. Second, we show how the desired self-manageability can be designed and implemented by means of a WSC manageability infrastructure. The presented approach is based on widely accepted methodologies and standards from the area of application and web service management, in particular the WBEM standards.

  @inproceedings{momm2008b, abstract = {Web service compositions (WSC), as part of a service-oriented architecture (SOA), have to be managed to ensure compliance with guaranteed service levels. In this context, a high degree of automation is desired, which can be achieved by applying autonomic computing concepts. This paper particularly focuses the autonomic management of semi-dynamic compositions. Here, for each included service several variants are available that differ with regard to the service level they offer. Given this scenario, we first show how to instrument WSC in order to allow a controlling of the service level through switching the employed service variant. Second, we show how the desired self-manageability can be designed and implemented by means of a WSC manageability infrastructure. The presented approach is based on widely accepted methodologies and standards from the area of application and web service management, in particular the WBEM standards.}, address = {Berlin, Germany}, author = {Momm, Christof and Rathfelder, Christoph}, booktitle = {MDD, SOA und IT-Management (MSI 2008)}, editor = {Steffens, Ulrike and Addicks, Jan Stefan and Streekmann, Niels}, keywords = {t_workshop}, month = {09}, pages = {25–40}, publisher = {GITO-Verlag}, title = {{M}odel-based {M}anagement of {W}eb {S}ervice {C}ompositions in {S}ervice-{O}riented {A}rchitectures}, year = 2008 }
• 
  QoS-aware resource allocation and load-balancing in enterprise Grids using online simulation. S. Kounev; (2008, August).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Oldenburg08, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), University of Oldenburg, Department of Computing Science, Oldenburg, Germany}, keywords = {Talks}, month = {08}, title = {{QoS-aware resource allocation and load-balancing in enterprise Grids using online simulation}}, year = 2008 }
• 
  Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28 K. Sachs; in Lecture Notes in Computer Science (LNCS), (S. Kounev; I. Gorton; K. Sachs, eds.) (2008). (Vol. 5119) Springer, Heidelberg, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.

  @proceedings{kounev2008c, abstract = {This book constitutes the refereed proceedings of the SPEC International Performance Evaluation Workshop, SIPEW 2008, held in Darmstadt, Germany, in June 2008. The 17 revised full papers presented together with 3 keynote talks were carefully reviewed and selected out of 39 submissions for inclusion in the book. The papers are organized in topical sections on models for software performance engineering; benchmarks and workload characterization; Web services and service-oriented architectures; power and performance; and profiling, monitoring and optimization.}, address = {Heidelberg, Germany}, author = {Sachs, Kai}, editor = {Kounev, Samuel and Gorton, Ian and Sachs, Kai}, keywords = {Power}, month = {06}, publisher = {Springer}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Performance Evaluation: Metrics, Models and Benchmarks, Proceedings of the 2008 SPEC International Performance Evaluation Workshop (SIPEW 2008), Darmstadt, Germany, June 27-28}}, volume = 5119, year = 2008 }
• 
  iSOAMM: An independent SOA Maturity Model. C. Rathfelder; H. Groenda; in Proceedings of the 8th IFIP International Conference on Distributed Applications and Interoperable Systems (DAIS 2008) (2008). (Vol. 5053/2008) 1–15.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  The implementation of an enterprise-wide Service Oriented Architecture (SOA) is a complex task. In most cases, evolutional approaches are used to handle this complexity. Maturity models are a possibility to plan and control such an evolution as they allow evaluating the current maturity and identifying current shortcomings. In order to support an SOA implementation, maturity models should also support in the selection of the most adequate maturity level and the deduction of a roadmap to this level. Existing SOA maturity models provide only weak assistance with the selection of an adequate maturity level. Most of them are developed by vendors of SOA products and often used to promote their products. In this paper, we introduce our independent SOA Maturity Model (iSOAMM), which is independent of the used technologies and products. In addition to the impacts on IT systems, it reflects the implications on organizational structures and governance. Furthermore, the iSOAMM lists the challenges, benefits and risks associated with each maturity level. This enables enterprises to select the most adequate maturity level for them, which is not necessarily the highest one.

  @inproceedings{rathfelder2008a, abstract = {The implementation of an enterprise-wide Service Oriented Architecture (SOA) is a complex task. In most cases, evolutional approaches are used to handle this complexity. Maturity models are a possibility to plan and control such an evolution as they allow evaluating the current maturity and identifying current shortcomings. In order to support an SOA implementation, maturity models should also support in the selection of the most adequate maturity level and the deduction of a roadmap to this level. Existing SOA maturity models provide only weak assistance with the selection of an adequate maturity level. Most of them are developed by vendors of SOA products and often used to promote their products. In this paper, we introduce our independent SOA Maturity Model (iSOAMM), which is independent of the used technologies and products. In addition to the impacts on IT systems, it reflects the implications on organizational structures and governance. Furthermore, the iSOAMM lists the challenges, benefits and risks associated with each maturity level. This enables enterprises to select the most adequate maturity level for them, which is not necessarily the highest one.}, address = {Berlin, Heidelberg}, author = {Rathfelder, Christoph and Groenda, Henning}, booktitle = {Proceedings of the 8th IFIP International Conference on Distributed Applications and Interoperable Systems (DAIS 2008)}, keywords = {se2}, month = {06}, pages = {1–15}, publisher = {Springer-Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {i{SOAMM}: {A}n independent {SOA} {M}aturity {M}odel}, volume = {5053/2008}, year = 2008 }

• 
  Autonomic QoS-Aware Resource Management in Grid Computing using Online Performance Models. S. Kounev; R. Nou; J. Torres; in Proceedings of the Second International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2007), Nantes, France, October 23-25, 2007 (2007). 1–10.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, an approach to autonomic QoS-aware resource management in Grid computing based on online performance models is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.

  @inproceedings{KoNoTo2007-VALUETOOLS-GridAuton_QoS_Control, abstract = {As Grid Computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. The inherent complexity, heterogeneity and dynamics of Grid computing environments pose some challenges in managing their capacity to ensure that QoS requirements are continuously met. In this paper, an approach to autonomic QoS-aware resource management in Grid computing based on online performance models is proposed. The paper presents a novel methodology for designing autonomic QoS-aware resource managers that have the capability to predict the performance of the Grid components they manage and allocate resources in such a way that service level agreements are honored. The goal is to make the Grid middleware self-configurable and adaptable to changes in the system environment and workload. The approach is subjected to an extensive experimental evaluation in the context of a real-world Grid environment and its effectiveness, practicality and performance are demonstrated.}, address = {Brussels, Belgium}, author = {Kounev, Samuel and Nou, Ramon and Torres, Jordi}, booktitle = {Proceedings of the Second International Conference on Performance Evaluation Methodologies and Tools (VALUETOOLS 2007), Nantes, France, October 23-25, 2007}, keywords = {Self-adaptive-systems}, pages = {1–10}, publisher = {ICST}, title = {{Autonomic QoS-Aware Resource Management in Grid Computing using Online Performance Models}}, year = 2007 }
• 
  QPME 1.0 User’s Guide S. Kounev; C. Dutz; (2007). Technische Universität Darmstadt, Darmstadt, Germany.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  This document describes the software package QPME (Queueing Petri net Modeling Environment), a performance modeling and analysis tool based on the Queueing Petri Net (QPN) modeling formalism. QPN models are more sophisticated than conventional queueing networks and stochastic Petri nets and have greater expressive power. This provides a number of important benefits since it makes it possible to model systems at a higher degree of accuracy. QPME is made of two components: QPE (QPN Editor) and SimQPN (Simulator for QPNs). QPE provides a user-friendly graphical tool for modeling using QPNs based on the Eclipse/GEF framework. SimQPN provides an efficient discrete-event simulation engine for QPNs that makes it possible to analyze models of realistically-sized systems. QPME runs on a wide range of platforms including Windows, Linux and Solaris. QPME is developed and maintained by Samuel Kounev and Christofer Dutz.

  @manual{Ko2007-TUD-QPME_UsersGuide, abstract = {This document describes the software package QPME (Queueing Petri net Modeling Environment), a performance modeling and analysis tool based on the Queueing Petri Net (QPN) modeling formalism. QPN models are more sophisticated than conventional queueing networks and stochastic Petri nets and have greater expressive power. This provides a number of important benefits since it makes it possible to model systems at a higher degree of accuracy. QPME is made of two components: QPE (QPN Editor) and SimQPN (Simulator for QPNs). QPE provides a user-friendly graphical tool for modeling using QPNs based on the Eclipse/GEF framework. SimQPN provides an efficient discrete-event simulation engine for QPNs that makes it possible to analyze models of realistically-sized systems. QPME runs on a wide range of platforms including Windows, Linux and Solaris. QPME is developed and maintained by Samuel Kounev and Christofer Dutz.}, address = {Darmstadt, Germany}, author = {Kounev, Samuel and Dutz, Christofer}, keywords = {QPN}, month = {01}, organization = {Technische Universit{{\"a}}t Darmstadt}, title = {{QPME 1.0 User's Guide}}, year = 2007 }
• 
  Designing a Workload Scenario for Benchmarking Message-Oriented Middleware. K. Sachs; S. Kounev; M. Carter; A. Buchmann; in Proceedings of the 2007 SPEC Benchmark Workshop, Austin, Texas, January 21, 2007 (2007).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Message-oriented middleware (MOM) is increasingly adopted as an enabling technology for modern information-driven applications like event-driven supply chain management, transport information monitoring, stock trading and online auctions to name just a few. There is a strong interest in the commercial and research domains for a standardized benchmark suite for evaluating the performance and scalability of MOM. With all major vendors adopting JMS (Java Message Service) as a standard interface to MOM servers, there is at last a means for creating a standardized workload for evaluating products in this space. This paper describes a novel application in the supply chain management domain that has been specifically designed as a representative workload scenario for evaluating the performance and scalability of MOM products. This scenario is used as a basis in SPEC's new SPECjms benchmark which will be the world's first industry-standard benchmark for MOM.

  @inproceedings{SaKoCaBu2007-SPEC_BW-BenchmarkingMOM, abstract = {Message-oriented middleware (MOM) is increasingly adopted as an enabling technology for modern information-driven applications like event-driven supply chain management, transport information monitoring, stock trading and online auctions to name just a few. There is a strong interest in the commercial and research domains for a standardized benchmark suite for evaluating the performance and scalability of MOM. With all major vendors adopting JMS (Java Message Service) as a standard interface to MOM servers, there is at last a means for creating a standardized workload for evaluating products in this space. This paper describes a novel application in the supply chain management domain that has been specifically designed as a representative workload scenario for evaluating the performance and scalability of MOM products. This scenario is used as a basis in SPEC's new SPECjms benchmark which will be the world's first industry-standard benchmark for MOM.}, author = {Sachs, Kai and Kounev, Samuel and Carter, Marc and Buchmann, Alejandro}, booktitle = {Proceedings of the 2007 SPEC Benchmark Workshop, Austin, Texas, January 21, 2007}, keywords = {t_workshop}, month = {01}, publisher = {SPEC}, title = {{Designing a Workload Scenario for Benchmarking Message-Oriented Middleware}}, year = 2007 }
• 
  Petri Net, Theory and Application. S. Kounev; A. Buchmann; V. Kordic (ed.) (2007).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Predictive performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. The challenge stems from the limited model expressiveness on the one hand and the limited scalability of model analysis techniques on the other. This chapter presents a novel methodology for modeling and performance analysis of distributed systems. The methodology is based on queueing Petri nets (QPNs) which provide greater modeling power and expressiveness than conventional modeling paradigms such as queueing networks and generalized stochastic Petri nets. Using QPNs, one can integrate both hardware and software aspects of system behavior into the same model. In addition to hardware contention and scheduling strategies, QPNs make it easy to model software contention, simultaneous resource possession, synchronization, blocking and asynchronous processing. These aspects have significant impact on the performance of modern distributed systems. To avoid the problem of state space explosion, our methodology uses discrete event simulation for model analysis. We propose an efficient and reliable method for simulation of QPNs. As a validation of our approach, we present a case study of a real-world distributed system, showing how our methodology is applied in a step-by-step fashion to evaluate the system performance and scalability. The system studied is a deployment of the industry-standard SPECjAppServer2004 benchmark. A detailed model of the system and its workload is built and used to predict the system performance for several deployment configurations and workload scenarios of interest. Taking advantage of the expressive power of QPNs, our approach makes it possible to model systems at a higher degree of accuracy providing a number of important benefits.

  @inbook{KoBu2007-ARS-OnTheUseOfQPNs, abstract = {Predictive performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. The challenge stems from the limited model expressiveness on the one hand and the limited scalability of model analysis techniques on the other. This chapter presents a novel methodology for modeling and performance analysis of distributed systems. The methodology is based on queueing Petri nets (QPNs) which provide greater modeling power and expressiveness than conventional modeling paradigms such as queueing networks and generalized stochastic Petri nets. Using QPNs, one can integrate both hardware and software aspects of system behavior into the same model. In addition to hardware contention and scheduling strategies, QPNs make it easy to model software contention, simultaneous resource possession, synchronization, blocking and asynchronous processing. These aspects have significant impact on the performance of modern distributed systems. To avoid the problem of state space explosion, our methodology uses discrete event simulation for model analysis. We propose an efficient and reliable method for simulation of QPNs. As a validation of our approach, we present a case study of a real-world distributed system, showing how our methodology is applied in a step-by-step fashion to evaluate the system performance and scalability. The system studied is a deployment of the industry-standard SPECjAppServer2004 benchmark. A detailed model of the system and its workload is built and used to predict the system performance for several deployment configurations and workload scenarios of interest. Taking advantage of the expressive power of QPNs, our approach makes it possible to model systems at a higher degree of accuracy providing a number of important benefits.}, address = {Vienna, Austria}, author = {Kounev, Samuel and Buchmann, Alejandro}, chapter = {On the Use of Queueing Petri Nets for Modeling and Performance Analysis of Distributed Systems}, editor = {Kordic, Vedran}, keywords = {QPN}, month = {02}, publisher = {Advanced Robotic Systems International, I-Tech Education and Publishing}, title = {Petri Net, Theory and Application}, year = 2007 }
• 
  An Approach to Performance Modeling and Analysis of Distributed Event-Based Systems. S. Kounev; (2007, March).
  • [ BibTeX ]
  @misc{talk-Kounev-Camb07, author = {Kounev, Samuel}, howpublished = {University of Cambridge, Computer Laboratory, Cambridge, UK}, keywords = {Talks}, month = {03}, title = {{An Approach to Performance Modeling and Analysis of Distributed Event-Based Systems}}, year = 2007 }
• 
  Management in serviceorientierten Architekturen: Eine Managementinfrastruktur für die Überwachung komponierter Webservices C. Rathfelder; (2007). VDM Verlag Dr. Müller, Saarbrücken, Germany.
  • [ BibTeX ]
  @proceedings{rathfelder2007a, address = {Saarbr{\"u}cken, Germany}, author = {Rathfelder, Christoph}, keywords = {t_book}, month = {04}, publisher = {VDM Verlag Dr. M{\"u}ller}, title = {{M}anagement in serviceorientierten {A}rchitekturen: {E}ine {M}anagementinfrastruktur f{\"u}r die {\"U}berwachung komponierter {W}ebservices}, year = 2007 }
• 
  Using QPN models for QoS Control in Grid Middleware. (UPC-DAC-RR-CAP-2007-4), S. Kounev; R. Nou; J. Torres; (2007).
  • [ BibTeX ]
  @techreport{KoNoTo2007-UPC-GridAutoQoS, author = {Kounev, Samuel and Nou, Ramon and Torres, Jordi}, institution = {Computer Architecture Department, Technical University of Catalonia (UPC), Spain}, keywords = {Self-adaptive-systems}, month = {04}, number = {UPC-DAC-RR-CAP-2007-4}, title = {{Using QPN models for QoS Control in Grid Middleware}}, year = 2007 }
• 
  Performance Modeling and Evaluation of Distributed Component Systems using Queueing Petri Nets. S. Kounev; (2007, May).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Edinburgh07, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), University of Edinburgh, Laboratory for Foundations of Computer Science, School of Informatics, Edinburgh, UK}, keywords = {Talks}, month = {05}, title = {{Performance Modeling and Evaluation of Distributed Component Systems using Queueing Petri Nets}}, year = 2007 }
• 
  SPEC Enterprise Java Benchmarks: State of the Art and Future Directions. S. Kounev; (2007, June).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-SPEC08, author = {Kounev, Samuel}, howpublished = {Invited Talk, SPEC Benchmarking Joint US/Europe Colloquium, Dresden, Germany}, keywords = {Talks}, month = {06}, title = {{SPEC Enterprise Java Benchmarks: State of the Art and Future Directions}}, year = 2007 }
• 
  Towards a Common API for Publish/Subscribe. P. Pietzuch; D. Eyers; S. Kounev; B. Shand; in Proceedings of the 2007 Inaugural International Conference on Distributed Event-Based Systems (DEBS 2007), Toronto, Canada, June 20-22, 2007, H.-A. Jacobsen, G. Mühl, M. A. Jaeger (eds.) (2007). (Vol. 233) 152–157.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Over the last decade a wide range of publish/subscribe (pub/sub) systems have come out of the research community. However, there is little consensus on a common pub/sub API, which would facilitate innovation, encourage application building, and simplify the evaluation of existing prototypes. Industry pub/sub standards tend to be overly complex, technology-centric, and hard to extend, thus limiting their applicability in research systems. In this paper we propose a common API for pub/sub that is tailored towards research requirements. The API supports three levels of compliance (with optional extensions): the lowest level specifies abstract operations without prescribing an implementation or data model; medium compliance describes interactions using a light-weight XML-RPC mechanism; finally, the highest level of compliance enforces an XML-RPC data model, enabling systems to understand each other's event and subscription semantics. We show that, by following this flexible approach with emphasis on extensibility, our API can be supported by many prototype systems with little effort.

  @inproceedings{PiEyKoSh2007-DEBS-PubSubAPI, abstract = {Over the last decade a wide range of publish/subscribe (pub/sub) systems have come out of the research community. However, there is little consensus on a common pub/sub API, which would facilitate innovation, encourage application building, and simplify the evaluation of existing prototypes. Industry pub/sub standards tend to be overly complex, technology-centric, and hard to extend, thus limiting their applicability in research systems. In this paper we propose a common API for pub/sub that is tailored towards research requirements. The API supports three levels of compliance (with optional extensions): the lowest level specifies abstract operations without prescribing an implementation or data model; medium compliance describes interactions using a light-weight XML-RPC mechanism; finally, the highest level of compliance enforces an XML-RPC data model, enabling systems to understand each other's event and subscription semantics. We show that, by following this flexible approach with emphasis on extensibility, our API can be supported by many prototype systems with little effort.}, author = {Pietzuch, Peter and Eyers, David and Kounev, Samuel and Shand, Brian}, booktitle = {Proceedings of the 2007 Inaugural International Conference on Distributed Event-Based Systems (DEBS 2007), Toronto, Canada, June 20-22, 2007}, editor = {Jacobsen, Hans-Arno and M{\"u}hl, Gero and Jaeger, Michael A.}, keywords = {descartes}, month = {06}, pages = {152–157}, publisher = {ACM, New York, NY, USA}, series = {ACM International Conference Proceeding Series}, title = {{Towards a Common API for Publish/Subscribe}}, volume = 233, year = 2007 }
• 
  A Manageability Infrastructure for the Monitoring of Web Service. C. Momm; C. Mayerl; C. Rathfelder; S. Abeck; in Proceedings of the 14th Annual Workshop of HP Software University Association, H. G. Hegering, H. Reiser, M. Schiffers, T. Nebe (eds.) (2007). 103–114.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The management of web service composition, where the employed atomic web services as well as the compositions themselves are offered on basis of Service Level Agreements (SLA), implies new requirements for the management infrastructure. In this paper we introduce the conceptual design and implementation for a standard-based and flexible manageability infrastructure offering comprehensive management information for an SLAdriven management of web service compositions. Our solution thereby is based on well-understood methodologies and standards from the area of application and web service management, in particular the WBEM standards.

  @inproceedings{momm2007a, abstract = {The management of web service composition, where the employed atomic web services as well as the compositions themselves are offered on basis of Service Level Agreements (SLA), implies new requirements for the management infrastructure. In this paper we introduce the conceptual design and implementation for a standard-based and flexible manageability infrastructure offering comprehensive management information for an SLAdriven management of web service compositions. Our solution thereby is based on well-understood methodologies and standards from the area of application and web service management, in particular the WBEM standards.}, address = {Stuttgart, Germany}, author = {Momm, Christof and Mayerl, Christian and Rathfelder, Christoph and Abeck, Sebastian}, booktitle = {Proceedings of the 14th Annual Workshop of HP Software University Association}, editor = {Hegering, H. G. and Reiser, H. and Schiffers, M. and Nebe, Th.}, keywords = {t_workshop}, month = {07}, pages = {103–114}, publisher = {Infonomies Consulting}, title = {{A} {M}anageability {I}nfrastructure for the {M}onitoring of {W}eb {S}ervice}, year = 2007 }
• 
  Workload Characterization of the SPECjms2007 Benchmark. K. Sachs; S. Kounev; J. Bacon; A. Buchmann; in Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27--28, 2007, K. Wolter (ed.) (2007). (Vol. 4748) 228–244.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. There is a strong interest in the end user and analyst communities for a standardized benchmark suite for evaluating the performance and scalability of MOM. In this paper, we present a workload characterization of the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. In addition to providing standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to customize the workload according to their requirements. The workload characterization presented in this paper serves two purposes i) to help users understand the internal components of the SPECjms2007 workload and the way they are scaled, ii) to show how the workload can be customized to exercise and evaluate selected aspects of MOM performance.We discuss how the various features supported by the benchmark can be exploited for in-depth performance analysis of MOM infrastructures.

  @inproceedings{SaKoBaBu2007-EPEW-WorkloadChar_SPECjms2007, abstract = {Message-oriented middleware (MOM) is at the core of a vast number of financial services and telco applications, and is gaining increasing traction in other industries, such as manufacturing, transportation, health-care and supply chain management. There is a strong interest in the end user and analyst communities for a standardized benchmark suite for evaluating the performance and scalability of MOM. In this paper, we present a workload characterization of the SPECjms2007 benchmark which is the world's first industry-standard benchmark specialized for MOM. In addition to providing standard workload and metrics for MOM performance, the benchmark provides a flexible performance analysis framework that allows users to customize the workload according to their requirements. The workload characterization presented in this paper serves two purposes i) to help users understand the internal components of the SPECjms2007 workload and the way they are scaled, ii) to show how the workload can be customized to exercise and evaluate selected aspects of MOM performance.We discuss how the various features supported by the benchmark can be exploited for in-depth performance analysis of MOM infrastructures.}, address = {Heidelberg, Germany}, author = {Sachs, Kai and Kounev, Samuel and Bacon, Jean and Buchmann, Alejandro}, booktitle = {Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27–28, 2007}, editor = {Wolter, Katinka}, keywords = {t_workshop}, month = {09}, pages = {228–244}, publisher = {Springer Verlag}, series = {Lecture Notes in Computer Science (LNCS)}, title = {{Workload Characterization of the SPECjms2007 Benchmark}}, volume = 4748, year = 2007 }
• 
  Building Online Performance Models of Grid Middleware with Fine-Grained Load-Balancing: A Globus Toolkit Case Study. R. Nou; S. Kounev; J. Torres; in Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27-28, 2007, K. Wolter (ed.) (2007). 125–140.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  As Grid computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. To guarantee that QoS requirements are continuously satisfied, the Grid middleware must be capable of predicting the application performance on the fly when deciding how to distribute the workload among the available resources. One way to achieve this is by using online performance models that get generated and analyzed on the fly. In this paper, we present a novel case study with the Globus Toolkit in which we show how performance models can be generated dynamically and used to provide online performance prediction capabilities. We have augmented the Grid middleware with an online performance prediction component that can be called at any time during operation to predict the Grid performance for a given resource allocation and load-balancing strategy. We evaluate the quality of our performance prediction mechanism and present some experimental results that demonstrate its effectiveness and practicality. The framework we propose can be used to design intelligent QoS-aware resource allocation and admission control mechanisms.

  @inproceedings{NoKoTo2007-EPEW-GridOnlinePerfModels, abstract = {As Grid computing increasingly enters the commercial domain, performance and Quality of Service (QoS) issues are becoming a major concern. To guarantee that QoS requirements are continuously satisfied, the Grid middleware must be capable of predicting the application performance on the fly when deciding how to distribute the workload among the available resources. One way to achieve this is by using online performance models that get generated and analyzed on the fly. In this paper, we present a novel case study with the Globus Toolkit in which we show how performance models can be generated dynamically and used to provide online performance prediction capabilities. We have augmented the Grid middleware with an online performance prediction component that can be called at any time during operation to predict the Grid performance for a given resource allocation and load-balancing strategy. We evaluate the quality of our performance prediction mechanism and present some experimental results that demonstrate its effectiveness and practicality. The framework we propose can be used to design intelligent QoS-aware resource allocation and admission control mechanisms.}, address = {Heidelberg, Germany}, author = {Nou, Ramon and Kounev, Samuel and Torres, Jordi}, booktitle = {Formal Methods and Stochastic Models for Performance Evaluation, Proceedings of the 4th European Performance Engineering Workshop (EPEW 2007), Berlin, Germany, September 27-28, 2007}, editor = {Wolter, Katinka}, keywords = {t_workshop}, month = {09}, number = 4748, pages = {125–140}, publisher = {Springer}, series = {Lecture Notes in Computer Science}, title = {{Building Online Performance Models of Grid Middleware with Fine-Grained Load-Balancing: A Globus Toolkit Case Study}}, year = 2007 }
• 
  SPECjms2007 - A Novel Benchmark and Performance Analysis Framework for Message-Oriented Middleware. S. Kounev; (2007, October).
  • [ BibTeX ]
  @misc{talk-Kounev-SPECjms07, author = {Kounev, Samuel}, howpublished = {University of Cambridge, Computer Laboratory, Cambridge, UK}, keywords = {Talks}, month = 10, title = {{SPECjms2007 - A Novel Benchmark and Performance Analysis Framework for Message-Oriented Middleware}}, year = 2007 }
• 
  Geschäftsprozessorientierte Kategorisierung von SOA. C. Rathfelder; H. Groenda; in 2. Workshop Bewertungsaspekte serviceorientierter Architekturen (2007). 11–22.
  • [ Abstract ]
  • [ BibTeX ]
  Service-Orientierte Architekturen (SOAs) versprechen eine bessere Unterstützung von Geschäftsprozessen. Es gibt jedoch unterschiedliche Interpretationen darüber, was eine Service-Orientierte Architektur (SOA) ist. Da die Verbesserung der Geschäftsprozessunterstützung eines der häufigsten Argumente für SOAs ist, bietet es sich an, die verschiedenen SOA-Varianten nach der damit ermöglichten Prozessunterstützung zu kategorisieren. Bisherige Ansätze zur Kategorisierung sind in vielen Fällen auf bestimmte Technologien oder Standards beschränkt und gehen nur am Rand auf die gegebene Prozessunterstützung ein. In diesem Artikel wird eine solche geschäftsprozessorientierte Kategorisierung von SOAs präsentiert.

  @inproceedings{rathfelder2007c, abstract = {Service-Orientierte Architekturen (SOAs) versprechen eine bessere Unterst{\"u}tzung von Gesch{\"a}ftsprozessen. Es gibt jedoch unterschiedliche Interpretationen dar{\"u}ber, was eine Service-Orientierte Architektur (SOA) ist. Da die Verbesserung der Gesch{\"a}ftsprozessunterst{\"u}tzung eines der h{\"a}ufigsten Argumente f{\"u}r SOAs ist, bietet es sich an, die verschiedenen SOA-Varianten nach der damit erm{\"o}glichten Prozessunterst{\"u}tzung zu kategorisieren. Bisherige Ans{\"a}tze zur Kategorisierung sind in vielen F{\"a}llen auf bestimmte Technologien oder Standards beschr{\"a}nkt und gehen nur am Rand auf die gegebene Prozessunterst{\"u}tzung ein. In diesem Artikel wird eine solche gesch{\"a}ftsprozessorientierte Kategorisierung von SOAs pr{\"a}sentiert.}, author = {Rathfelder, Christoph and Groenda, Henning}, booktitle = {2. Workshop Bewertungsaspekte serviceorientierter Architekturen}, keywords = {t_workshop}, month = 11, pages = {11–22}, publisher = {SHAKER Verlag}, title = {{G}esch{\"a}ftsprozessorientierte {K}ategorisierung von {SOA}}, year = 2007 }
• 
  Preliminary Analysis of Globus Toolkit 4 to Create Prediction Models. (UPC-DAC-RR-2007-37), R. Nou; S. Kounev; (2007).
  • [ BibTeX ]
  @techreport{NoKo2007-UPC-GlobusOnlinePerfModels, author = {Nou, Ramon and Kounev, Samuel}, institution = {Computer Architecture Department, Technical University of Catalonia (UPC), Spain}, keywords = {QPME_Bibliography}, month = {07}, number = {UPC-DAC-RR-2007-37}, title = {{Preliminary Analysis of Globus Toolkit 4 to Create Prediction Models}}, year = 2007 }

• 
  Message Types and Interfaces Between Components in SPECjms. (DVS06-3), K. Sachs; S. Kounev; (2006).
  • [ BibTeX ]
  @techreport{SaKo2006-MessageTypesInSPECjms, author = {Sachs, Kai and Kounev, Samuel}, institution = {SPEC OSG Java Subcommittee}, keywords = {SPEC}, month = {02}, number = {DVS06-3}, title = {{Message Types and Interfaces Between Components in SPECjms}}, year = 2006 }
• 
  Towards a Manageability Infrastructure for a Management of Process-Based Service Compositions C. Momm; C. Rathfelder; S. Abeck; (2006).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  The management of process-oriented service composition within a dynamic environment, where the employed core services are offered on service marketplaces and dynamically included into the composition on basis of Service Level Agreements (SLA), demands for a service management application that takes into account the specifics of processoriented compositions and supports their automated provisioning. As a first step towards such an application, in this paper we introduce the conceptual design for an architecture and implementation of an interoperable and flexible manageability infrastructure offering comprehensive monitoring and control functionality for the management of service compositions. To achieve this, our approach is based on well-understood methodologies and standards from the area of application and web service management.

  @techreport{momm2006a, abstract = {The management of process-oriented service composition within a dynamic environment, where the employed core services are offered on service marketplaces and dynamically included into the composition on basis of Service Level Agreements (SLA), demands for a service management application that takes into account the specifics of processoriented compositions and supports their automated provisioning. As a first step towards such an application, in this paper we introduce the conceptual design for an architecture and implementation of an interoperable and flexible manageability infrastructure offering comprehensive monitoring and control functionality for the management of service compositions. To achieve this, our approach is based on well-understood methodologies and standards from the area of application and web service management.}, author = {Momm, Christof and Rathfelder, Christoph and Abeck, Sebastian}, institution = {Cooperation {&} Management}, keywords = {se2}, title = {{T}owards a {M}anageability {I}nfrastructure for a {M}anagement of {P}rocess-{B}ased {S}ervice {C}ompositions}, type = {C{&}M Research Report}, year = 2006 }
• 
  Workload Scenario for SPECjms - Supermarket Supply Chain. (DVS06-2), K. Sachs; S. Kounev; (2006).
  • [ Abstract ]
  • [ BibTeX ]
  Message-oriented middleware (MOM) is increasingly adopted as an enabling technology for modern informationdriven applications like event-driven supply chain management, transport information monitoring, stock trading and online auctions to name just a few. There is a strong interest in the commercial and research domains for a standardized benchmark suite for evaluating the performance and scalability of MOM. With all major vendors adopting JMS (Java Message Service) as a standard interface to MOM servers, there is at last a means for creating a standardized workload for evaluating products in this space. This paper describes a novel application in the supply chain management domain that has been specifically designed as a representative workload scenario for evaluating the performance and scalability of MOM products. This scenario is used as a basis in SPEC?s new SPECjms benchmark which will be the world?s first industry-standard benchmark for MOM.

  @techreport{SaKo2006-WorkloadScenarioSPECjms, abstract = {Message-oriented middleware (MOM) is increasingly adopted as an enabling technology for modern informationdriven applications like event-driven supply chain management, transport information monitoring, stock trading and online auctions to name just a few. There is a strong interest in the commercial and research domains for a standardized benchmark suite for evaluating the performance and scalability of MOM. With all major vendors adopting JMS (Java Message Service) as a standard interface to MOM servers, there is at last a means for creating a standardized workload for evaluating products in this space. This paper describes a novel application in the supply chain management domain that has been specifically designed as a representative workload scenario for evaluating the performance and scalability of MOM products. This scenario is used as a basis in SPEC?s new SPECjms benchmark which will be the world?s first industry-standard benchmark for MOM.}, author = {Sachs, Kai and Kounev, Samuel}, institution = {SPEC OSG Java Subcommittee}, keywords = {SPEC}, month = {01}, number = {DVS06-2}, title = {{Workload Scenario for SPECjms - Supermarket Supply Chain}}, year = 2006 }
• 
  J2EE Performance and Scalability - From Measuring to Predicting. S. Kounev; in Proceedings of the 2006 SPEC Benchmark Workshop, Austin, Texas, USA (2006).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  J2EE applications are becoming increasingly ubiquitous and with their increasing adoption, performance and scalability issues are gaining in importance. For a J2EE application to perform well and be scalable, both the platform on which it is built and the application design must be efficient and scalable. Industry-standard benchmarks such as the SPECjAppServer set of benchmarks help to evaluate the performance and scalability of alternative platforms for J2EE applications, however, they cannot be used to evaluate the performance and scalability of concrete applications built on the selected platforms. In this paper, we present a systematic approach for evaluating and predicting the performance and scalability of J2EE applications based on modeling and simulation. The approach helps to identify and eliminate bottlenecks in the application design and ensure that systems are designed and sized to meet their quality of service requirements. We introduce our approach by showing how it can be applied to the SPECjAppServer2004 benchmark which is used as a representative J2EE application. A detailed model of a SPECjAppServer2004 deployment is built in a step-by-step fashion and then used to predict the behavior of the system under load. The approach is validated by comparing model predictions against measurements on the real system.

  @inproceedings{Ko2006-SPEC_BW-J2EEPerfScal, abstract = {J2EE applications are becoming increasingly ubiquitous and with their increasing adoption, performance and scalability issues are gaining in importance. For a J2EE application to perform well and be scalable, both the platform on which it is built and the application design must be efficient and scalable. Industry-standard benchmarks such as the SPECjAppServer set of benchmarks help to evaluate the performance and scalability of alternative platforms for J2EE applications, however, they cannot be used to evaluate the performance and scalability of concrete applications built on the selected platforms. In this paper, we present a systematic approach for evaluating and predicting the performance and scalability of J2EE applications based on modeling and simulation. The approach helps to identify and eliminate bottlenecks in the application design and ensure that systems are designed and sized to meet their quality of service requirements. We introduce our approach by showing how it can be applied to the SPECjAppServer2004 benchmark which is used as a representative J2EE application. A detailed model of a SPECjAppServer2004 deployment is built in a step-by-step fashion and then used to predict the behavior of the system under load. The approach is validated by comparing model predictions against measurements on the real system.}, author = {Kounev, Samuel}, booktitle = {Proceedings of the 2006 SPEC Benchmark Workshop, Austin, Texas, USA}, keywords = {t_workshop}, month = {01}, title = {{J2EE Performance and Scalability - From Measuring to Predicting}}, year = 2006 }
• 
  Case Studies of Queueing Networks --- J2EE Applications. S. Kounev; in Queueing Networks and Markov Chains, edited by Gunter Bolch, Stefan Greiner, Hermann de Meer and Kishor Shridharbhai Trivedi (2006). (2ndnd ed.) 733–745.
  • [ BibTeX ]
  • [ URL ]
  @inbook{KoBu2006-WileyInterscience-QN_CaseStudy_J2EE, author = {Kounev, Samuel}, booktitle = {{Queueing Networks and Markov Chains, edited by Gunter Bolch, Stefan Greiner, Hermann de Meer and Kishor Shridharbhai Trivedi}}, edition = {2nd}, keywords = {t_bookchapter}, month = {04}, pages = {733–745}, publisher = {Wiley-Interscience, John Wiley & Sons Inc.}, title = {{Case Studies of Queueing Networks — J2EE Applications}}, year = 2006 }
• 
  Performance Modeling and Evaluation of Event-based Systems. S. Kounev; (2006, March).
  • [ BibTeX ]
  @misc{talk-Kounev-Camb06-2, author = {Kounev, Samuel}, howpublished = {University of Cambridge, Computer Laboratory, Cambridge, UK}, keywords = {Talks}, month = {03}, title = {{Performance Modeling and Evaluation of Event-based Systems}}, year = 2006 }
• 
  A Practical Approach for Predicting the Performance and Scalability of Distributed Component Systems. S. Kounev; (2006, April).
  • [ BibTeX ]
  @misc{talk-Kounev-Camb06, author = {Kounev, Samuel}, howpublished = {University of Cambridge, Computer Laboratory, Cambridge, UK}, keywords = {Talks}, month = {04}, title = {{A Practical Approach for Predicting the Performance and Scalability of Distributed Component Systems}}, year = 2006 }
• 
  SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation. S. Kounev; A. Buchmann; in Performance Evaluation (2006). 63(4-5) 364–394.
  • [ BibTeX ]
  • [ Download ]
  @article{KoBu2006-PERFEVAL-SimQPN, address = {Amsterdam, The Netherlands}, author = {Kounev, Samuel and Buchmann, Alejandro}, journal = {Performance Evaluation}, keywords = {QPN}, month = {05}, number = {4-5}, pages = {364–394}, publisher = {Elsevier Science Publishers B. V.}, title = {{SimQPN - a tool and methodology for analyzing queueing Petri net models by means of simulation}}, volume = 63, year = 2006 }
• 
  Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets. S. Kounev; in IEEE Transactions on Software Engineering (2006). 32(7) 486–502.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.

  @article{Ko2006-IEEE_TSE-QPN_ModelingMethod, abstract = {Performance models are used increasingly throughout the phases of the software engineering lifecycle of distributed component-based systems. However, as systems grow in size and complexity, building models that accurately capture the different aspects of their behavior becomes a more and more challenging task. In this paper, we present a novel case study of a realistic distributed component-based system, showing how Queueing Petri Net models can be exploited as a powerful performance prediction tool in the software engineering process. A detailed system model is built in a step-by-step fashion, validated, and then used to evaluate the system performance and scalability. Along with the case study, a practical performance modeling methodology is presented which helps to construct models that accurately reflect the system performance and scalability characteristics. Taking advantage of the modeling power and expressiveness of Queueing Petri Nets, our approach makes it possible to model the system at a higher degree of accuracy, providing a number of important benefits.}, author = {Kounev, Samuel}, journal = {IEEE Transactions on Software Engineering}, keywords = {QPME_Bibliography}, month = {07}, number = 7, pages = {486–502}, title = {{Performance Modeling and Evaluation of Distributed Component-Based Systems using Queueing Petri Nets}}, volume = 32, year = 2006 }
• 
  QPME - Queueing Petri Net Modeling Environment. S. Kounev; C. Dutz; A. Buchmann; in Proceedings of the 3rd International Conference on Quantitative Evaluation of SysTems (QEST 2006), Riverside, California, USA, September 11-14, 2006 (2006). 115–116.
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. However, currently available tools for modeling and analysis using queueing Petri nets are very limited in terms of the scalability of the analysis algorithms they provide. Moreover, tools are available only on highly specialized platforms unaccessible to most potential users. In this paper, we present QPME - a Queueing Petri Net Modeling Environment that supports the modeling and analysis of systems using queueing Petri nets. QPME runs on a wide range of platforms and provides a powerful simulation engine that can be used to analyze models of realistically-sized systems.

  @inproceedings{KoDuBu2006-QEST-QPME, abstract = {Queueing Petri nets are a powerful formalism that can be exploited for modeling distributed systems and analyzing their performance and scalability. However, currently available tools for modeling and analysis using queueing Petri nets are very limited in terms of the scalability of the analysis algorithms they provide. Moreover, tools are available only on highly specialized platforms unaccessible to most potential users. In this paper, we present QPME - a Queueing Petri Net Modeling Environment that supports the modeling and analysis of systems using queueing Petri nets. QPME runs on a wide range of platforms and provides a powerful simulation engine that can be used to analyze models of realistically-sized systems.}, address = {Washington, DC, USA}, author = {Kounev, Samuel and Dutz, Christofer and Buchmann, Alejandro}, booktitle = {Proceedings of the 3rd International Conference on Quantitative Evaluation of SysTems (QEST 2006), Riverside, California, USA, September 11-14, 2006}, keywords = {QPN}, month = {09}, pages = {115–116}, publisher = {IEEE Computer Society}, title = {{QPME - Queueing Petri Net Modeling Environment}}, year = 2006 }

• 
  Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction. S. Kounev; Thesis; Technische Universität Darmstadt, Germany. (2005, December).

  Distinguished Dissertation Award
  • [ BibTeX ]
  • [ Download ]
  @phdthesis{Ko2005-TUD-PhD_Thesis_as_phdthesis_class, author = {Kounev, Samuel}, keywords = {SPEC}, month = 12, note = {<b>Distinguished Dissertation Award</b>}, school = {Technische Universit{\"a}t Darmstadt, Germany}, title = {Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction}, year = 2005 }
• 
  Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction S. Kounev; (2005). Shaker Verlag, Ph.D. Thesis, Technische Universität Darmstadt, Germany.

  Distinguished Dissertation Award
  • [ BibTeX ]
  • [ Download ]
  @book{Ko2005-TUD-PhD_Thesis, author = {Kounev, Samuel}, keywords = {QPME_Bibliography}, month = 12, note = {<b>Distinguished Dissertation Award</b>}, publisher = {Shaker Verlag, Ph.D. Thesis, Technische Universit{\"a}t Darmstadt, Germany}, title = {Performance Engineering of Distributed Component-Based Systems - Benchmarking, Modeling and Performance Prediction}, year = 2005 }
• 
  SPECjAppServer2004 - The New Way to Evaluate J2EE Performance. S. Kounev; (2005, September).
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  This article presents SPECjAppServer2004---the new industry-standard benchmark for measuring the performance and scalability of J2EE hardware and software platforms. SPECjAppServer2004 is a completely new benchmark and not comparable to the SPEC J2EE benchmarks released in late 2002. This article discusses the business domains and workload modeled by the benchmark, as well as the benchmark design and architecture. The author also explains the meaning of the benchmark metrics, discusses the different purposes the benchmark can be used, and provides some links to additional information.

  @misc{Ko2005-DEV2DEV-SPECjAppServer2004, abstract = {This article presents SPECjAppServer2004—the new industry-standard benchmark for measuring the performance and scalability of J2EE hardware and software platforms. SPECjAppServer2004 is a completely new benchmark and not comparable to the SPEC J2EE benchmarks released in late 2002. This article discusses the business domains and workload modeled by the benchmark, as well as the benchmark design and architecture. The author also explains the meaning of the benchmark metrics, discusses the different purposes the benchmark can be used, and provides some links to additional information.}, author = {Kounev, Samuel}, howpublished = {DEV2DEV Article, O'Reilly Publishing Group}, keywords = {SPEC}, month = {09}, title = {{SPECjAppServer2004 - The New Way to Evaluate J2EE Performance}}, year = 2005 }
• 
  Performance Engineering of Distributed Component Systems - From Benchmarking to Modeling and Performance Prediction. S. Kounev; (2005, April).
  • [ BibTeX ]
  @misc{talk-Kounev-TUBerlin05, author = {Kounev, Samuel}, howpublished = {Invited Talk, Technical University of Berlin, Institut f{\"u}r Technische Informatik und Mikroelektronik, Berlin}, keywords = {Talks}, month = {04}, title = {{Performance Engineering of Distributed Component Systems - From Benchmarking to Modeling and Performance Prediction}}, year = 2005 }
• 
  E-Business Performance and Scalability - From Measuring to Predicting. S. Kounev; (2005, January).
  • [ BibTeX ]
  @misc{talk-Kounev-SAP05, author = {Kounev, Samuel}, howpublished = {Invited Talk (Kolloquiumsvortrag), SAP Research Forum, Walldorf, Germany}, keywords = {Talks}, month = {01}, title = {{E-Business Performance and Scalability - From Measuring to Predicting}}, year = 2005 }

• 
  Performance Tuning and Optimization of J2EE Applications on the JBoss Platform. S. Kounev; B. Weis; A. Buchmann; in Journal of Computer Resource Management (2004). 113
  • [ BibTeX ]
  • [ Download ]
  @article{KoWeBu2004-CMG_JCRM-JBoss, author = {Kounev, Samuel and Weis, B{\"o}rn and Buchmann, Alejandro}, journal = {Journal of Computer Resource Management}, keywords = {Optimization}, month = {09}, publisher = {Computer Measurement Group (CMG)}, title = {{Performance Tuning and Optimization of J2EE Applications on the JBoss Platform}}, volume = 113, year = 2004 }

• 
  Performance Modeling of Distributed E-Business Applications using Queueing Petri Nets. S. Kounev; A. Buchmann; in Proceedings of the 2003 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS 2003), Austin, Texas, USA, March 6-8, 2003 (2003). 143–155.

  Best-Paper-Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  In this paper we show how Queuing Petri Net (QPN) models can be exploited for performance analysis of distributed e-business systems. We study a real-world application, and demonstrate the benefits, in terms of modelling power and expressiveness, that QPN models provide over conventional modelling paradigms such as Queuing Networks and Petri Nets. As shown, QPNs facilitate the integration of both hardware and software aspects of system behavior in the same model. In addition to hardware contention and scheduling strategies, using QPNs one can easily model simultaneous resource possession, synchronization, blocking and contention for software resources. By validating the models presented through measurements, we show that they are not just powerful as a specification mechanism, but are also very powerful as a performance analysis and prediction tool. However, currently available tools and techniques for QPN analysis are limited. Improved solution methods, which enable larger models to be analyzed, need to be developed. By demonstrating the power of QPNs as a modelling paradigm in realistic scenarios, we hope to motivate further research in this area.

  @inproceedings{KoBu2003-ISPASS-ModelingUsingQPNs, abstract = {In this paper we show how Queuing Petri Net (QPN) models can be exploited for performance analysis of distributed e-business systems. We study a real-world application, and demonstrate the benefits, in terms of modelling power and expressiveness, that QPN models provide over conventional modelling paradigms such as Queuing Networks and Petri Nets. As shown, QPNs facilitate the integration of both hardware and software aspects of system behavior in the same model. In addition to hardware contention and scheduling strategies, using QPNs one can easily model simultaneous resource possession, synchronization, blocking and contention for software resources. By validating the models presented through measurements, we show that they are not just powerful as a specification mechanism, but are also very powerful as a performance analysis and prediction tool. However, currently available tools and techniques for QPN analysis are limited. Improved solution methods, which enable larger models to be analyzed, need to be developed. By demonstrating the power of QPNs as a modelling paradigm in realistic scenarios, we hope to motivate further research in this area.}, address = {Washington, DC, USA}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 2003 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS 2003), Austin, Texas, USA, March 6-8, 2003}, keywords = {QPN}, month = {03}, note = {<b>Best-Paper-Award</b>}, pages = {143–155}, publisher = {IEEE Computer Society}, title = {Performance Modeling of Distributed E-Business Applications using Queueing Petri Nets}, year = 2003 }
• 
  Performance Modeling of Distributed E-Business Applications using Queueing Petri Nets. S. Kounev; (2003, June).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-Dagstuhl03, author = {Kounev, Samuel}, howpublished = {Invited Talk, Dagstuhl - Workshop of five Informatik-Graduiertenkolles, Schloss Dagstuhl}, keywords = {Talks}, month = {06}, title = {{Performance Modeling of Distributed E-Business Applications using Queueing Petri Nets}}, year = 2003 }
• 
  Messaging Architecture and Asynchronous Interactions in SPECjAppServer. (TUD03-1), S. Kounev; (2003).
  • [ BibTeX ]
  @techreport{Ko2003-SPEC-SPECjAppServer, author = {Kounev, Samuel}, institution = {SPEC OSG Java Subcommittee}, keywords = {Optimization}, month = {09}, number = {TUD03-1}, title = {{Messaging Architecture and Asynchronous Interactions in SPECjAppServer}}, year = 2003 }
• 
  Performance Modeling and Evaluation of Large-Scale J2EE Applications. S. Kounev; A. Buchmann; in Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003 (2003). 273–283.

  Best-Paper-Award
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Modern J2EE applications are typically based on highly distributed architectures comprising multiple components deployed in a clustered environment. This makes it difficult for deployers to estimate the capacity of the deployment environment needed to guarantee that Service Level Agreements are met. This paper looks at the different approaches to this problem and discusses the difficulties that arise when one tries to apply them to large, real-world systems. The authors study a realistic J2EE application (the SPECjAppServer2002 benchmark) and show how analytical models can be exploited for capacity planning.

  @inproceedings{KoBu2003-CMG-PerfModeling_J2EEApps, abstract = {Modern J2EE applications are typically based on highly distributed architectures comprising multiple components deployed in a clustered environment. This makes it difficult for deployers to estimate the capacity of the deployment environment needed to guarantee that Service Level Agreements are met. This paper looks at the different approaches to this problem and discusses the difficulties that arise when one tries to apply them to large, real-world systems. The authors study a realistic J2EE application (the SPECjAppServer2002 benchmark) and show how analytical models can be exploited for capacity planning.}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = 12, note = {<b>Best-Paper-Award</b>}, organization = {Computer Measurement Group (CMG)}, pages = {273–283}, title = {{Performance Modeling and Evaluation of Large-Scale J2EE Applications}}, year = 2003 }
• 
  PetStore-WS: Measuring the Performance Implications of Web Services. K. S. Juse; S. Kounev; A. Buchmann; in Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003 (2003). 113–123.
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  • [ Download ]
  Web Services are increasingly used to enable loosely coupled integration among heterogeneous systems but are perceived as a source of severe performance degradation. This paper looks at the impact on system performance when introducing Web Service interfaces to an originally tightly coupled application. Using two implementation variants of Sun's Java Pet Store application, one based strictly on the J2EE platform and the other implementing some interfaces as Web Services, performance is compared in terms of the achieved overall throughput, response times and latency.

  @inproceedings{JuKoBu2003-CMG-PetStoreWS, abstract = {Web Services are increasingly used to enable loosely coupled integration among heterogeneous systems but are perceived as a source of severe performance degradation. This paper looks at the impact on system performance when introducing Web Service interfaces to an originally tightly coupled application. Using two implementation variants of Sun's Java Pet Store application, one based strictly on the J2EE platform and the other implementing some interfaces as Web Services, performance is compared in terms of the achieved overall throughput, response times and latency.}, author = {Juse, Kai S. and Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 29th International Conference of the Computer Measurement Group on Resource Management and Performance Evaluation of Enterprise Computing Systems (CMG 2003), Dallas, Texas, USA, December 7-12, 2003}, keywords = {Optimization}, month = 12, organization = {Computer Measurement Group (CMG)}, pages = {113–123}, title = {{PetStore-WS: Measuring the Performance Implications of Web Services}}, year = 2003 }

• 
  J2EE PERFORMANCE: From Benchmarking to Sizing and Capacity Planning. S. Kounev; (2002, May).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-DVSPhD02, author = {Kounev, Samuel}, howpublished = {DVS-PhD Internal Workshop, Darmstadt, Germany}, keywords = {Talks}, month = {05}, title = {{J2EE PERFORMANCE: From Benchmarking to Sizing and Capacity Planning}}, year = 2002 }
• 
  Performance Issues in E-Business Systems. S. Kounev; A. Buchmann; in Proceedings of the International Conference on Advances in Infrastructure for e-Business, e-Education, e-Science, and e-Medicine on the Internet (SSGRR 2002w), L’Aquila, Italy, January 21-27, 2002 (2002).
  • [ Abstract ]
  • [ BibTeX ]
  • [ Download ]
  Performance and scalability issues in e-business systems are gaining in importance as we move from hype and prototypes to real operational systems. Typical for this development is also the emergence of standard benchmarks of which TPC-W for transactional B2C systems and ECperf for performance and scalability measurement of application servers are two of the better known examples. In this paper we present an experience report with the ECperf benchmark defined by Sun and discuss performance issues that we observed in our implementation of the benchmark. Some of these issues are related to the specification of the benchmark, for which we made suggestions how to correct them and others are related to database connectivity, locking patterns, and the need for asynchronous processing.

  @inproceedings{KoBu2002-SSGRR-PerfIssuesEB, abstract = {Performance and scalability issues in e-business systems are gaining in importance as we move from hype and prototypes to real operational systems. Typical for this development is also the emergence of standard benchmarks of which TPC-W for transactional B2C systems and ECperf for performance and scalability measurement of application servers are two of the better known examples. In this paper we present an experience report with the ECperf benchmark defined by Sun and discuss performance issues that we observed in our implementation of the benchmark. Some of these issues are related to the specification of the benchmark, for which we made suggestions how to correct them and others are related to database connectivity, locking patterns, and the need for asynchronous processing.}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the International Conference on Advances in Infrastructure for e-Business, e-Education, e-Science, and e-Medicine on the Internet (SSGRR 2002w), L'Aquila, Italy, January 21-27, 2002}, keywords = {Optimization}, month = {01}, title = {{Performance Issues in E-Business Systems}}, year = 2002 }
• 
  Improving Data Access of J2EE Applications by Exploiting Asynchronous Messaging and Caching Services. S. Kounev; A. Buchmann; in Proceedings of the 28th International Conference on Very Large Data Bases (VLDB 2002), Hong Kong, China, August 20--23, 2002 (2002). 574–585.

  Acceptance Rate (Full Paper): 14% Best-Paper-Award Nomination
  • [ Abstract ]
  • [ BibTeX ]
  • [ URL ]
  The J2EE platform provides a variety of options for making business data persistent using DBMS technology. However, the integration with existing backend database systems has proven to be of crucial importance for the scalability and performance of J2EE applications, because modern e-business systems are extremely data-intensive. As a result, the data access layer, and the link between the application server and the database server in particular, are very susceptible to turning into a system bottleneck. In this paper we use the ECperf benchmark as an example of a realistic application in order to illustrate the problems mentioned above and discuss how they could be approached and eliminated. In particular, we show how asynchronous, message-based processing could be exploited to reduce the load on the DBMS and improve system performance, scalability and reliability. Furthermore, we discuss the major issues related to the correct use of entity beans (the components provided by J2EE for modelling persistent data) and present a number of methods to optimize their performance utilizing caching mechanisms. We have evaluated the proposed techniques through measurements and have documented the performance gains that they provide.

  @inproceedings{KoBu2002-VLDB-ImprovingDataAccessJ2EE, abstract = {The J2EE platform provides a variety of options for making business data persistent using DBMS technology. However, the integration with existing backend database systems has proven to be of crucial importance for the scalability and performance of J2EE applications, because modern e-business systems are extremely data-intensive. As a result, the data access layer, and the link between the application server and the database server in particular, are very susceptible to turning into a system bottleneck. In this paper we use the ECperf benchmark as an example of a realistic application in order to illustrate the problems mentioned above and discuss how they could be approached and eliminated. In particular, we show how asynchronous, message-based processing could be exploited to reduce the load on the DBMS and improve system performance, scalability and reliability. Furthermore, we discuss the major issues related to the correct use of entity beans (the components provided by J2EE for modelling persistent data) and present a number of methods to optimize their performance utilizing caching mechanisms. We have evaluated the proposed techniques through measurements and have documented the performance gains that they provide.}, author = {Kounev, Samuel and Buchmann, Alejandro}, booktitle = {Proceedings of the 28th International Conference on Very Large Data Bases (VLDB 2002), Hong Kong, China, August 20–23, 2002}, keywords = {Optimization}, note = {Acceptance Rate (Full Paper): 14\% <b>Best-Paper-Award Nomination</b>}, pages = {574–585}, publisher = {VLDB Endowment, Morgan Kaufmann}, title = {{Improving Data Access of J2EE Applications by Exploiting Asynchronous Messaging and Caching Services}}, year = 2002 }

• 
  Implementing and Optimizing Sun’s ECperf Benchmark with BEA WebLogic Server. S. Kounev; (2001, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-BEA01, author = {Kounev, Samuel}, howpublished = {Invited Talk, BEA Hochschul-Industrie-Kooperations-Tagung, Frankfurt, Germany}, keywords = {Talks}, month = 11, title = {{Implementing and Optimizing Sun's ECperf Benchmark with BEA WebLogic Server}}, year = 2001 }
• 
  Eliminating ECperf Persistence Bottlenecks when using RDBMS with Pessimistic Concurrency Control S. Kounev; (2001).
  • [ BibTeX ]
  • [ Download ]
  @techreport{Ko2001-SUN-ECperf, author = {Kounev, Samuel}, institution = {ECperf Expert Group at Sun Microsystems Inc.}, keywords = {Optimization}, month = {09}, title = {{Eliminating ECperf Persistence Bottlenecks when using RDBMS with Pessimistic Concurrency Control}}, year = 2001 }
• 
  A Capacity Planning Methodology for Distributed E-Commerce Applications S. Kounev; (2001).
  • [ BibTeX ]
  • [ Download ]
  @techreport{Ko2001-TUD-CapPlanMethod, author = {Kounev, Samuel}, institution = {Technische Universit{{\"a}}t Darmstadt, Germany}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = {02}, title = {{A Capacity Planning Methodology for Distributed E-Commerce Applications}}, year = 2001 }
• 
  Performance Prediction, Sizing and Capacity Planning for Distributed E-Commerce Applications S. Kounev; (2001).
  • [ BibTeX ]
  • [ Download ]
  @techreport{Ko2001-TUD-PerfPred, author = {Kounev, Samuel}, institution = {Technische Universit{{\"a}}t Darmstadt, Germany}, keywords = {Instrumentation_profiling_and_workload_characterization}, month = {01}, title = {{Performance Prediction, Sizing and Capacity Planning for Distributed E-Commerce Applications}}, year = 2001 }

• 
  Electronic Store: MALL2000 - Mall for Online Business beyond the Year 2000. S. Kounev; (1999, November).
  • [ BibTeX ]
  • [ Download ]
  @misc{talk-Kounev-BGOUG99, author = {Kounev, Samuel}, howpublished = {Invited Talk, Bulgarian Oracle User Group (BGOUG) Annual Conference 1999, Sofia, Bulgaria}, keywords = {Talks}, month = 11, title = {{Electronic Store: MALL2000 - Mall for Online Business beyond the Year 2000}}, year = 1999 }
• 
  The Analysis Phase in the Development of E-Commerce Software Systems. S. Kounev; K. Nikolov; in Proceedings of the Tools Eastern Europe ’99 Conference on Technology of Object Oriented Languages and Systems, Sofia-Blagoevgrad, Bulgaria, June 1-4, 1999 (1999).
  • [ BibTeX ]
  @inproceedings{KoNi1999-TOOLS_EE-Analysis_Phase, author = {Kounev, Samuel and Nikolov, Kiril}, booktitle = {Proceedings of the Tools Eastern Europe '99 Conference on Technology of Object Oriented Languages and Systems, Sofia-Blagoevgrad, Bulgaria, June 1-4, 1999}, keywords = {descartes}, month = {06}, title = {{The Analysis Phase in the Development of E-Commerce Software Systems}}, year = 1999 }
• 
  Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server. P. Nenov; S. Kounev; D. Mihailov; in Journal of Computing and Information (1999).
  • [ BibTeX ]
  @article{NeKoMi1999-JCI-VideoConf, author = {Nenov, Plamen and Kounev, Samuel and Mihailov, Dimiter}, journal = {Journal of Computing and Information}, keywords = {t_journalmagazine}, title = {{Distributed Video-Conferencing System Organized for Work on the Internet with the Use of Multimedia Server}}, year = 1999 }