Publications

{Integrating Statistical Response Time Models in Architectural Performance Models}. Eismann, Simon; Grohmann, Johannes; Walter, J{ü}rgen; von Kistowski, J{ó}akim; Kounev, Samuel; in Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA) (2019). 71–80. IEEE.

Acceptance Rate: 21,9\% (21/96)

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ DOI ]
[ Download ]

Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.

@inproceedings{EiGrWaKiKo2019-ICSA-Integrating,
  abstract = {Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.},
  author = {Eismann, Simon and Grohmann, Johannes and Walter, J{\"u}rgen and von Kistowski, J{\'o}akim and Kounev, Samuel},
  booktitle = {Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA)},
  keywords = {PRISMA},
  month = {03},
  note = {Acceptance Rate: 21,9\% (21/96)},
  pages = {71–80},
  publisher = {IEEE},
  title = {{Integrating Statistical Response Time Models in Architectural Performance Models}},
  year = 2019
}

%0 Conference Paper
%1 EiGrWaKiKo2019-ICSA-Integrating
%A Eismann, Simon
%A Grohmann, Johannes
%A Walter, J{ü}rgen
%A von Kistowski, J{ó}akim
%A Kounev, Samuel
%B Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA)
%D 2019
%I IEEE
%P 71--80
%R 10.1109/ICSA.2019.00016
%T {Integrating Statistical Response Time Models in Architectural Performance Models}
%U https://doi.org/10.1109/ICSA.2019.00016
%X Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.

{How is Performance Addressed in DevOps?}. Bezemer, Cor{-}Paul; Eismann, Simon; Ferme, Vincenzo; Grohmann, Johannes; Heinrich, Robert; Jamshidi, Pooyan; Shang, Weiyi; van Hoorn, Andr{{é}}; Villavicencio, M{{ó}}nica; Walter, J{{ü}}rgen; Willnecker, Felix; in Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (2019). 45–50. Association for Computing Machinery (ACM), New York, NY, USA.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ DOI ]
[ Download ]

DevOps is a modern software engineering paradigm that is gaining widespread adoption in industry. The goal of DevOps is to bring software changes into production with a high frequency and fast feedback cycles. This conflicts with software quality assurance activities, particularly with respect to performance. For instance, performance evaluation activities --- such as load testing --- require a considerable amount of time to get statistically significant results. We conducted an industrial survey to get insights into how performance is addressed in industrial DevOps settings. In particular, we were interested in the frequency of executing performance evaluations, the tools being used, the granularity of the obtained performance data, and the use of model-based techniques. The survey responses, which come from a wide variety of participants from different industry sectors, indicate that the complexity of performance engineering approaches and tools is a barrier for wide-spread adoption of performance analysis in DevOps. The implication of our results is that performance analysis tools need to have a short learning curve, and should be easy to integrate into the DevOps pipeline in order to be adopted by practitioners.

@inproceedings{BeEiFeGrRhJaShHoViWaWi2019-ICPE-DevOpsSurvey,
  abstract = {DevOps is a modern software engineering paradigm that is gaining widespread adoption in industry. The goal of DevOps is to bring software changes into production with a high frequency and fast feedback cycles. This conflicts with software quality assurance activities, particularly with respect to performance. For instance, performance evaluation activities — such as load testing — require a considerable amount of time to get statistically significant results. We conducted an industrial survey to get insights into how performance is addressed in industrial DevOps settings. In particular, we were interested in the frequency of executing performance evaluations, the tools being used, the granularity of the obtained performance data, and the use of model-based techniques. The survey responses, which come from a wide variety of participants from different industry sectors, indicate that the complexity of performance engineering approaches and tools is a barrier for wide-spread adoption of performance analysis in DevOps. The implication of our results is that performance analysis tools need to have a short learning curve, and should be easy to integrate into the DevOps pipeline in order to be adopted by practitioners.},
  address = {New York, NY, USA},
  author = {Bezemer, Cor{-}Paul and Eismann, Simon and Ferme, Vincenzo and Grohmann, Johannes and Heinrich, Robert and Jamshidi, Pooyan and Shang, Weiyi and van Hoorn, Andr{{\'e}} and Villavicencio, M{{\'o}}nica and Walter, J{{\"u}}rgen and Willnecker, Felix},
  booktitle = {Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering},
  keywords = {SPEC},
  pages = {45–50},
  publisher = {Association for Computing Machinery (ACM)},
  series = {ICPE '19},
  title = {{How is Performance Addressed in DevOps?}},
  year = 2019
}

%0 Conference Paper
%1 BeEiFeGrRhJaShHoViWaWi2019-ICPE-DevOpsSurvey
%A Bezemer, Cor{-}Paul
%A Eismann, Simon
%A Ferme, Vincenzo
%A Grohmann, Johannes
%A Heinrich, Robert
%A Jamshidi, Pooyan
%A Shang, Weiyi
%A van Hoorn, Andr{{é}}
%A Villavicencio, M{{ó}}nica
%A Walter, J{{ü}}rgen
%A Willnecker, Felix
%B Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering
%C New York, NY, USA
%D 2019
%I Association for Computing Machinery (ACM)
%P 45--50
%R 10.1145/3297663.3309672
%T {How is Performance Addressed in DevOps?}
%U https://doi.org/10.1145/3297663.3309672
%X DevOps is a modern software engineering paradigm that is gaining widespread adoption in industry. The goal of DevOps is to bring software changes into production with a high frequency and fast feedback cycles. This conflicts with software quality assurance activities, particularly with respect to performance. For instance, performance evaluation activities --- such as load testing --- require a considerable amount of time to get statistically significant results. We conducted an industrial survey to get insights into how performance is addressed in industrial DevOps settings. In particular, we were interested in the frequency of executing performance evaluations, the tools being used, the granularity of the obtained performance data, and the use of model-based techniques. The survey responses, which come from a wide variety of participants from different industry sectors, indicate that the complexity of performance engineering approaches and tools is a barrier for wide-spread adoption of performance analysis in DevOps. The implication of our results is that performance analysis tools need to have a short learning curve, and should be easy to integrate into the DevOps pipeline in order to be adopted by practitioners.
%@ 9781450362399

Automation in Software Performance Engineering Based on a Declarative Specification of Concerns. Walter, J{ü}rgen Christian; (2019). Universit{ä}t W{ü}rzburg.

[ BibTeX ]
[ EndNote ]
[ URL ]

@phdthesis{Walter2018-phd,
  author = {Walter, J{\"u}rgen Christian},
  keywords = {PMX},
  school = {Universit{\"a}t W{\"u}rzburg},
  title = {Automation in Software Performance Engineering Based on a Declarative Specification of Concerns},
  type = {doctoralthesis},
  year = 2019
}

%0 Thesis
%1 Walter2018-phd
%A Walter, J{ü}rgen Christian
%D 2019
%T Automation in Software Performance Engineering Based on a Declarative Specification of Concerns
%U https://opus.bibliothek.uni-wuerzburg.de/frontdoor/index/index/docId/18090

Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking. Iffländer, Lukas; Geißler, Stefan; Walter, Jürgen; Beierlieb, Lukas; Kounev, Samuel; in Proceedings of the 9th Symposium on Software Performance 2018 (SSP’18) (2018).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Mooreâ€™s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

@inproceedings{Ifflaender2018_AddressingShortcomingsofExistingDDoSProtectionSoftwareUsingSoftware-DefinedNetworking,
  abstract = {Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Mooreâ€™s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.},
  author = {Iffländer, Lukas and Geißler, Stefan and Walter, Jürgen and Beierlieb, Lukas and Kounev, Samuel},
  booktitle = {Proceedings of the 9th Symposium on Software Performance 2018 (SSP'18)},
  keywords = {t_workshop},
  month = 11,
  title = {Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking},
  year = 2018
}

%0 Conference Paper
%1 Ifflaender2018_AddressingShortcomingsofExistingDDoSProtectionSoftwareUsingSoftware-DefinedNetworking
%A Iffländer, Lukas
%A Geißler, Stefan
%A Walter, Jürgen
%A Beierlieb, Lukas
%A Kounev, Samuel
%B Proceedings of the 9th Symposium on Software Performance 2018 (SSP'18)
%D 2018
%T Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking
%X Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Mooreâ€™s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

{Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time}. Eismann, Simon; Walter, J{ü}rgen; von Kistowski, J{ó}akim; Kounev, Samuel; in 2018 IEEE International Conference on Software Architecture (ICSA) (2018). 135–144.

Acceptance Rate: 25,6\% (22/86)

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.

@inproceedings{EiWaKiKo2018-ParametricDependencies,
  abstract = {Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.},
  author = {Eismann, Simon and Walter, J{\"u}rgen and von Kistowski, J{\'o}akim and Kounev, Samuel},
  booktitle = {2018 IEEE International Conference on Software Architecture (ICSA)},
  keywords = {Meta-models},
  month = {04},
  note = {Acceptance Rate: 25,6\% (22/86)},
  pages = {135-144},
  title = {{Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time}},
  year = 2018
}

%0 Conference Paper
%1 EiWaKiKo2018-ParametricDependencies
%A Eismann, Simon
%A Walter, J{ü}rgen
%A von Kistowski, J{ó}akim
%A Kounev, Samuel
%B 2018 IEEE International Conference on Software Architecture (ICSA)
%D 2018
%P 135-144
%T {Modeling of Parametric Dependencies for Performance Prediction of Component-based Software Systems at Run-time}
%U https://ieeexplore.ieee.org/document/8417146
%X Model-based performance analysis can be leveraged to explore performance properties of software systems. To capture the behavior of varying workload mixes, configurations, and deployments of a software system requires formal modeling of the impact of configuration parameters and user input on the system behavior. Such influences are represented as parametric dependencies in software performance models. Existing modeling approaches focus on modeling parametric dependencies at design-time. This paper identifies runtime specific parametric dependency features, which are not supported by existing work. Therefore, this paper proposes a novel modeling methodology for parametric dependencies and a corresponding graph-based resolution algorithm. This algorithm enables the solution of models containing component instance-level dependencies, variables with multiple descriptions in parallel, and correlations modeled as parametric dependencies. We integrate our work into the Descartes Modeling Language (DML), allowing for accurate and efficient modeling and analysis of parametric dependencies. These performance predictions are valuable for various purposes such as capacity planning, bottleneck analysis, configuration optimization and proactive auto-scaling. Our evaluation analyzes a video store application. The prediction for varying language mixes and video sizes shows a mean error below 5% for utilization and below 10% for response time.

{Tools for Declarative Performance Engineering}. Walter, J{ü}rgen; Eismann, Simon; Grohmann, Johannes; Okanovic, Dusan; Kounev, Samuel; in Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 53–56. Association for Computing Machinery (ACM), New York, NY, USA.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ DOI ]
[ Download ]

Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.

@inproceedings{WaEiGrOkKo2018-ICPE-Tools-for-DPE-Tutorial,
  abstract = {Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.},
  address = {New York, NY, USA},
  author = {Walter, J{\"u}rgen and Eismann, Simon and Grohmann, Johannes and Okanovic, Dusan and Kounev, Samuel},
  booktitle = {Companion of the 2018 ACM/SPEC International Conference on Performance Engineering},
  keywords = {DQL},
  pages = {53–56},
  publisher = {Association for Computing Machinery (ACM)},
  series = {ICPE '18},
  title = {{Tools for Declarative Performance Engineering}},
  year = 2018
}

%0 Conference Paper
%1 WaEiGrOkKo2018-ICPE-Tools-for-DPE-Tutorial
%A Walter, J{ü}rgen
%A Eismann, Simon
%A Grohmann, Johannes
%A Okanovic, Dusan
%A Kounev, Samuel
%B Companion of the 2018 ACM/SPEC International Conference on Performance Engineering
%C New York, NY, USA
%D 2018
%I Association for Computing Machinery (ACM)
%P 53--56
%R 10.1145/3185768.3185777
%T {Tools for Declarative Performance Engineering}
%U https://doi.org/10.1145/3185768.3185777
%X Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.
%@ 9781450356299

The Vision of Self-aware Reordering of Security Network Function Chains. Iffländer, Lukas; Walter, Jürgen; Eismann, Simon; Kounev, Samuel; in Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 1–4. ACM, New York, NY, USA.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.

@inproceedings{IfWaEiKo2018-ICPE-SSFC-Vision,
  abstract = {Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.},
  address = {New York, NY, USA},
  author = {Iffländer, Lukas and Walter, Jürgen and Eismann, Simon and Kounev, Samuel},
  booktitle = {Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering},
  keywords = {Self-adaptive-systems},
  pages = {1–4},
  publisher = {ACM},
  series = {ICPE '18},
  title = {The Vision of Self-aware Reordering of Security Network Function Chains},
  year = 2018
}

%0 Conference Paper
%1 IfWaEiKo2018-ICPE-SSFC-Vision
%A Iffländer, Lukas
%A Walter, Jürgen
%A Eismann, Simon
%A Kounev, Samuel
%B Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering
%C New York, NY, USA
%D 2018
%I ACM
%P 1--4
%T The Vision of Self-aware Reordering of Security Network Function Chains
%U https://dl.acm.org/doi/10.1145/3185768.3186309
%X Services provided online are subject to various types of attacks. Security appliances can be chained to protect a system against multiple types of network attacks. The sequence of appliances has a significant impact on the efficiency of the whole chain. While the operation of security appliance chains is currently based on a static order, traffic-aware reordering of security appliances may significantly improve efficiency and accuracy. In this paper, we present the vision a self-aware system to automatically reorder security appliances according to incoming traffic. To achieve this, we propose to apply a model-based learning, reasoning, and acting (LRA-M) loop. To this end, we describe a corresponding system architecture and explain its building blocks.

{Automated and Adaptable Decision Support for Software Performance Engineering}. Walter, J{ü}rgen; van Hoorn, Andre; Kounev, Samuel; in Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools (2017).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.

@inproceedings{WalterHoKo2017-DecisionSupport,
  abstract = {Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.},
  author = {Walter, J{\"u}rgen and van Hoorn, Andre and Kounev, Samuel},
  booktitle = {Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools},
  keywords = {DQL},
  month = 12,
  title = {{Automated and Adaptable Decision Support for Software Performance Engineering}},
  year = 2017
}

%0 Conference Paper
%1 WalterHoKo2017-DecisionSupport
%A Walter, J{ü}rgen
%A van Hoorn, Andre
%A Kounev, Samuel
%B Proceedings of the 11th EAI International Conference on Performance Evaluation Methodologies and Tools
%D 2017
%T {Automated and Adaptable Decision Support for Software Performance Engineering}
%X Software performance engineering (SPE) provides a plethora of methods and tooling for measuring, modeling, and evaluating performance properties of software systems. The solution approaches come with different strengths and limitations concerning, for example, accuracy, time-to-result, or system overhead. While approaches allow for interchangeability, the choice of an appropriate approach and tooling to solve a given performance concern still relies on expert knowledge. Currently, there is no automated and extensible approach for decision support. In this paper, we present a methodology for the automated selection of performance engineering approaches tailored to user concerns. We decouple the complexity of selecting an SPE approach for a given scenario providing a decision engine and solution approach capability models. This separation allows to easily append additional solution approaches and rating criteria. We demonstrate the applicability by presenting decision engines that compare measurement- and model-based analysis approaches.

{Automating the Build Pipeline for Docker Container}. Reed, Nikolai; Walter, J{ü}rgen; Kounev, Samuel; (2017, November).

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{ReWaKo2017-SPP-DockerPipeline,
  author = {Reed, Nikolai and Walter, J{\"u}rgen and Kounev, Samuel},
  howpublished = {Technical Talk at Symposion on Software Performance (SSP) 2017},
  keywords = {descartes},
  month = 11,
  title = {{Automating the Build Pipeline for Docker Container}},
  year = 2017
}

%0 Generic
%1 ReWaKo2017-SPP-DockerPipeline
%A Reed, Nikolai
%A Walter, J{ü}rgen
%A Kounev, Samuel
%D 2017
%T {Automating the Build Pipeline for Docker Container}

{Providing Model-Extraction-as-a-Service for Architectural Performance Models}. Walter, J{ü}rgen; Eismann, Simon; Reed, Nikolai; Kounev, Samuel; in Proceedings of the 2017 Symposium on Software Performance (SSP) (2017).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.

@inproceedings{WaEiReKo2017-SPP-MEaaS,
  abstract = {Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.},
  author = {Walter, J{\"u}rgen and Eismann, Simon and Reed, Nikolai and Kounev, Samuel},
  booktitle = {Proceedings of the 2017 Symposium on Software Performance (SSP)},
  keywords = {t_workshop},
  month = 11,
  title = {{Providing Model-Extraction-as-a-Service for Architectural Performance Models}},
  year = 2017
}

%0 Conference Paper
%1 WaEiReKo2017-SPP-MEaaS
%A Walter, J{ü}rgen
%A Eismann, Simon
%A Reed, Nikolai
%A Kounev, Samuel
%B Proceedings of the 2017 Symposium on Software Performance (SSP)
%D 2017
%T {Providing Model-Extraction-as-a-Service for Architectural Performance Models}
%X Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.

{Mapping of Service Level Objectives to Performance Queries}. Walter, J{ü}rgen; Okanovic, Dusan; Kounev, Samuel; in {Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)} (2017). ACM.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.

@inproceedings{WaOkKo2017-WOSPC-SLO2Queries,
  abstract = {The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.},
  author = {Walter, J{\"u}rgen and Okanovic, Dusan and Kounev, Samuel},
  booktitle = {{Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}},
  keywords = {DQL},
  month = {04},
  publisher = {ACM},
  title = {{Mapping of Service Level Objectives to Performance Queries}},
  year = 2017
}

%0 Conference Paper
%1 WaOkKo2017-WOSPC-SLO2Queries
%A Walter, J{ü}rgen
%A Okanovic, Dusan
%A Kounev, Samuel
%B {Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}
%D 2017
%I ACM
%T {Mapping of Service Level Objectives to Performance Queries}
%X The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.

{An Expandable Extraction Framework for Architectural Performance Models}. Walter, J{ü}rgen; Stier, Christian; Koziolek, Heiko; Kounev, Samuel; in {Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS’17)} (2017). ACM.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.

@inproceedings{WaStKoKo2017-QUDOS-PMXBuilder,
  abstract = {Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.},
  author = {Walter, J{\"u}rgen and Stier, Christian and Koziolek, Heiko and Kounev, Samuel},
  booktitle = {{Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17)}},
  keywords = {t_workshop},
  month = {04},
  publisher = {ACM},
  title = {{An Expandable Extraction Framework for Architectural Performance Models}},
  year = 2017
}

%0 Conference Paper
%1 WaStKoKo2017-QUDOS-PMXBuilder
%A Walter, J{ü}rgen
%A Stier, Christian
%A Koziolek, Heiko
%A Kounev, Samuel
%B {Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17)}
%D 2017
%I ACM
%T {An Expandable Extraction Framework for Architectural Performance Models}
%X Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.

CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches. D{ü}llmann, Thomas F.; Heinrich, Robert; van Hoorn, Andre; Pitakrat, Teerat; Walter, J{ü}rgen; Willnecker, Felix; in Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017) (2017). IEEE.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.

@inproceedings{DuHeHoPiWaWi2017-CASPA,
  abstract = {Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.},
  author = {D{\"u}llmann, Thomas F. and Heinrich, Robert and van Hoorn, Andre and Pitakrat, Teerat and Walter, J{\"u}rgen and Willnecker, Felix},
  booktitle = {Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017)},
  keywords = {PMX},
  publisher = {IEEE},
  title = {CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches},
  year = 2017
}

%0 Conference Paper
%1 DuHeHoPiWaWi2017-CASPA
%A D{ü}llmann, Thomas F.
%A Heinrich, Robert
%A van Hoorn, Andre
%A Pitakrat, Teerat
%A Walter, J{ü}rgen
%A Willnecker, Felix
%B Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017)
%D 2017
%I IEEE
%T CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches
%U http://eprints.uni-kiel.de/37422/
%X Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.

{Online Learning of Run-time Models for Performance and Resource Management in Data Centers}. Walter, J{ü}rgen; Marco, Antinisca Di; Spinner, Simon; Inverardi, Paola; Kounev, Samuel; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.

@incollection{WaMaSpInKo-learning,
  abstract = {In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.},
  address = {{Berlin Heidelberg, Germany}},
  author = {Walter, J{\"u}rgen and Marco, Antinisca Di and Spinner, Simon and Inverardi, Paola and Kounev, Samuel},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {Self-adaptive-systems},
  publisher = {{Springer Verlag}},
  title = {{Online Learning of Run-time Models for Performance and Resource Management in Data Centers}},
  year = 2017
}

%0 Book Section
%1 WaMaSpInKo-learning
%A Walter, J{ü}rgen
%A Marco, Antinisca Di
%A Spinner, Simon
%A Inverardi, Paola
%A Kounev, Samuel
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Online Learning of Run-time Models for Performance and Resource Management in Data Centers}
%X In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.

{Spacecraft Autonomous Reaction Capabilities, Control Approaches and Self-Aware Computing}. Schilling, Klaus; Walter, J{ü}rgen; Kounev, Samuel; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Space exploration missions require very challenging autonomous reaction capabilities, as spacecraft have to react appropriately to the partially unknown environment in time critical situations, where direct human interaction is often impossible due to significant signal propagation delays related to the huge distances. In particular, interplanetary exploration missions are especially demanding; therefore the missions CASSINI-HUYGEN (landing on the Saturnian moon) and ROSETTA (the accompanying and landing on a comet) will be emphasized. Today also distributed, networked, small satellites for telecommunication and Earth observation applications promise a paradigm shift compared to the traditional multi-functional big spacecraft and could benefit from self-aware computing capabilities.

@incollection{ScWaKo-space,
  abstract = {Space exploration missions require very challenging autonomous reaction capabilities, as spacecraft have to react appropriately to the partially unknown environment in time critical situations, where direct human interaction is often impossible due to significant signal propagation delays related to the huge distances. In particular, interplanetary exploration missions are especially demanding; therefore the missions CASSINI-HUYGEN (landing on the Saturnian moon) and ROSETTA (the accompanying and landing on a comet) will be emphasized. Today also distributed, networked, small satellites for telecommunication and Earth observation applications promise a paradigm shift compared to the traditional multi-functional big spacecraft and could benefit from self-aware computing capabilities.},
  address = {{Berlin Heidelberg, Germany}},
  author = {Schilling, Klaus and Walter, J{\"u}rgen and Kounev, Samuel},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {Self-adaptive-systems},
  publisher = {{Springer Verlag}},
  title = {{Spacecraft Autonomous Reaction Capabilities, Control Approaches and Self-Aware Computing}},
  year = 2017
}

%0 Book Section
%1 ScWaKo-space
%A Schilling, Klaus
%A Walter, J{ü}rgen
%A Kounev, Samuel
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Spacecraft Autonomous Reaction Capabilities, Control Approaches and Self-Aware Computing}
%X Space exploration missions require very challenging autonomous reaction capabilities, as spacecraft have to react appropriately to the partially unknown environment in time critical situations, where direct human interaction is often impossible due to significant signal propagation delays related to the huge distances. In particular, interplanetary exploration missions are especially demanding; therefore the missions CASSINI-HUYGEN (landing on the Saturnian moon) and ROSETTA (the accompanying and landing on a comet) will be emphasized. Today also distributed, networked, small satellites for telecommunication and Earth observation applications promise a paradigm shift compared to the traditional multi-functional big spacecraft and could benefit from self-aware computing capabilities.

{ Kieker4DQL: Declarative Performance Measurement}. Blohm, Matthias; Pahlberg, Maksim; Vogel, Sebastian; Walter, J{ü}rgen; Okanovic, Dusan; in Proceedings of the 2016 Symposium on Software Performance (SSP) (2016).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

The Descartes Query Language (DQL) enables to query the performance of a system using adapters to various solution approaches. Thereby, DQL is a realization of the vision of Declarative Performance Engineering (DPE) which decouples the description of user concerns (performance questions and goals) from the task of selecting and applying a speciffic solution approach. While model-based approaches are already supported by DQL, measurement based approaches are not. In this paper we present the Kieker4DQL, an adapter for DQL that performs query processing for Application Monitoring Tools, the Kieker in particular. It creates a tailored monitoring configuration to start measurements. The resulting monitoring data is filtered and interpreted according to the query.

@inproceedings{BlPhVoWaOk2016-SPP-Kieker4DQL,
  abstract = {The Descartes Query Language (DQL) enables to query the performance of a system using adapters to various solution approaches. Thereby, DQL is a realization of the vision of Declarative Performance Engineering (DPE) which decouples the description of user concerns (performance questions and goals) from the task of selecting and applying a speciffic solution approach. While model-based approaches are already supported by DQL, measurement based approaches are not. In this paper we present the Kieker4DQL, an adapter for DQL that performs query processing for Application Monitoring Tools, the Kieker in particular. It creates a tailored monitoring configuration to start measurements. The resulting monitoring data is filtered and interpreted according to the query.},
  author = {Blohm, Matthias and Pahlberg, Maksim and Vogel, Sebastian and Walter, J{\"u}rgen and Okanovic, Dusan},
  booktitle = {Proceedings of the 2016 Symposium on Software Performance (SSP)},
  keywords = {DQL},
  month = 11,
  title = {{ Kieker4DQL: Declarative Performance Measurement}},
  year = 2016
}

%0 Conference Paper
%1 BlPhVoWaOk2016-SPP-Kieker4DQL
%A Blohm, Matthias
%A Pahlberg, Maksim
%A Vogel, Sebastian
%A Walter, J{ü}rgen
%A Okanovic, Dusan
%B Proceedings of the 2016 Symposium on Software Performance (SSP)
%D 2016
%T { Kieker4DQL: Declarative Performance Measurement}
%X The Descartes Query Language (DQL) enables to query the performance of a system using adapters to various solution approaches. Thereby, DQL is a realization of the vision of Declarative Performance Engineering (DPE) which decouples the description of user concerns (performance questions and goals) from the task of selecting and applying a speciffic solution approach. While model-based approaches are already supported by DQL, measurement based approaches are not. In this paper we present the Kieker4DQL, an adapter for DQL that performs query processing for Application Monitoring Tools, the Kieker in particular. It creates a tailored monitoring configuration to start measurements. The resulting monitoring data is filtered and interpreted according to the query.

{ PAVO: A Framework for the Visualization of Performance Analyses Results}. Walter, J{ü}rgen; K{ö}nig, Maximilian; Eismann, Simon; Kounev, Samuel; in Proceedings of the 2016 Symposium on Software Performance (SSP) (2016).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).

@inproceedings{WaKoEiKo2016-SPP-PAVO,
  abstract = {Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).},
  author = {Walter, J{\"u}rgen and K{\"o}nig, Maximilian and Eismann, Simon and Kounev, Samuel},
  booktitle = {Proceedings of the 2016 Symposium on Software Performance (SSP)},
  keywords = {DQL},
  month = 11,
  title = {{ PAVO: A Framework for the Visualization of Performance Analyses Results}},
  year = 2016
}

%0 Conference Paper
%1 WaKoEiKo2016-SPP-PAVO
%A Walter, J{ü}rgen
%A K{ö}nig, Maximilian
%A Eismann, Simon
%A Kounev, Samuel
%B Proceedings of the 2016 Symposium on Software Performance (SSP)
%D 2016
%T { PAVO: A Framework for the Visualization of Performance Analyses Results}
%X Awareness of application performance can be derived through various quantitative analyses, model-based and measurement-based approaches. A suitable visualization supports the understanding of application performance. Usually, analysis tools include a tool specific visualization or no visualization at all. In this paper we proposed to decouple the result visualization from the analysis approach. We present the Performance VisualizatiOn (PAVO) framework which provides result visualization tailored to a given performance analysis result. To show benefits and usability of PAVO, we present our integration in Descartes Query Language (DQL).

{Automation and Simplification Through Declarative Performance Engineering}. Walter, J{ü}rgen; (2016, June). W{ü}rzburg, Germany.

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{Talk-Kolloquium-2016-06-14,
  address = {W{\"u}rzburg, Germany},
  author = {Walter, J{\"u}rgen},
  howpublished = {Doctoral Colloquium W{\"u}rzburg, June 14, 2016},
  keywords = {Optimization},
  month = {06},
  title = {{Automation and Simplification Through Declarative Performance Engineering}},
  year = 2016
}

%0 Generic
%1 Talk-Kolloquium-2016-06-14
%A Walter, J{ü}rgen
%C W{ü}rzburg, Germany
%D 2016
%T {Automation and Simplification Through Declarative Performance Engineering}

{Asking ``What?’’, Automating the ``How?’’: The Vision of Declarative Performance Engineering}. Walter, J{ü}rgen; van Hoorn, Andre; Koziolek, Heiko; Okanovic, Dusan; Kounev, Samuel; in {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)} (2016).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

@inproceedings{WaHoKoOkKo2016-ICPE-DPE_Vision,
  abstract = {Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.},
  author = {Walter, J{\"u}rgen and van Hoorn, Andre and Koziolek, Heiko and Okanovic, Dusan and Kounev, Samuel},
  booktitle = {{Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}},
  keywords = {DQL},
  month = {03},
  title = {{Asking ``What?'', Automating the ``How?'': The Vision of Declarative Performance Engineering}},
  year = 2016
}

%0 Conference Paper
%1 WaHoKoOkKo2016-ICPE-DPE_Vision
%A Walter, J{ü}rgen
%A van Hoorn, Andre
%A Koziolek, Heiko
%A Okanovic, Dusan
%A Kounev, Samuel
%B {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}
%D 2016
%T {Asking ``What?'', Automating the ``How?'': The Vision of Declarative Performance Engineering}
%X Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

{The Vision of Declarative Performance Engineering}. Walter, J{ü}rgen; van Hoorn, Andre; Koziolek, Heiko; Okanovic, Dusan; Kounev, Samuel; in {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)} (2016).

[ Abstract ]
[ BibTeX ]
[ EndNote ]

Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

@inproceedings{WaHoKoOkKo2016-ICPE-DPE_Poster,
  abstract = {Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.},
  author = {Walter, J{\"u}rgen and van Hoorn, Andre and Koziolek, Heiko and Okanovic, Dusan and Kounev, Samuel},
  booktitle = {{Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}},
  keywords = {DQL},
  month = {03},
  title = {{The Vision of Declarative Performance Engineering}},
  year = 2016
}

%0 Conference Paper
%1 WaHoKoOkKo2016-ICPE-DPE_Poster
%A Walter, J{ü}rgen
%A van Hoorn, Andre
%A Koziolek, Heiko
%A Okanovic, Dusan
%A Kounev, Samuel
%B {Proceedings of the 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}
%D 2016
%T {The Vision of Declarative Performance Engineering}
%X Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of ``what'' versus ``how'' enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

{A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}. Spinner, Simon; Walter, J{ü}rgen; Kounev, Samuel; in {Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)} (2016).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.

@inproceedings{SpWaKo2015-WOSPC-ReferenceArchitecture,
  abstract = {Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.},
  author = {Spinner, Simon and Walter, J{\"u}rgen and Kounev, Samuel},
  booktitle = {{Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}},
  keywords = {t_workshop},
  month = {03},
  title = {{A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}},
  year = 2016
}

%0 Conference Paper
%1 SpWaKo2015-WOSPC-ReferenceArchitecture
%A Spinner, Simon
%A Walter, J{ü}rgen
%A Kounev, Samuel
%B {Proceedings of the 2016 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'16) co-located with 7th ACM/SPEC International Conference on Performance Engineering (ICPE 2016)}
%D 2016
%T {A Reference Architecture for Online Performance Model Extraction in Virtualized Environments}
%X Performance models can support decisions throughout the life-cycle of a software system. However, the manual construction of such performance models is a complex and time-consuming task requiring deep system knowledge. Therefore, automatic approaches for creating and updating performance models of a running system are necessary. Existing work focuses on single aspects of model extraction or proposes approaches specifically designed for a certain technology stack. In virtualized environments, we often see different applications based on diverse technology stacks sharing the same infrastructure. In order to enable online performance model extraction in such environments, we describe a new reference architecture for integrating different specialized model extraction solutions.

{DECLARE: Declarative Performance Engineering}. van Hoorn, Andr{é}; Kounev, Samuel; Okanovi{{\’c}}, Dusan; Walter, J{ü}rgen; (2016, January). Hannover, Germany.

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{Talk-vanHoKo-SPP1593-2016,
  address = {Hannover, Germany},
  author = {van Hoorn, Andr{\'e} and Kounev, Samuel and Okanovi{\'c}, Dusan and Walter, J{\"u}rgen},
  howpublished = {Renewal Kickoff Workshop of the DFG Priority Programme 1593, Hannover, January 14-15, 2016},
  keywords = {Talks},
  month = {01},
  title = {{DECLARE: Declarative Performance Engineering}},
  year = 2016
}

%0 Generic
%1 Talk-vanHoKo-SPP1593-2016
%A van Hoorn, Andr{é}
%A Kounev, Samuel
%A Okanovi{{\'c}}, Dusan
%A Walter, J{ü}rgen
%C Hannover, Germany
%D 2016
%T {DECLARE: Declarative Performance Engineering}

{Automated Transformation of Descartes Modeling Language to Palladio Component Model}. Walter, J{ü}rgen; Eismann, Simon; Hildebrandt, Adrian; in Proceedings of the 2015 Symposium on Software Performance (SSP) (2015).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Model-based performance predictions and reconfigurations enable optimizing resource efficiency while ensuring that Quality-of-Service demands are met in today's complex ITsystems. The Descartes Modeling Language (DML) and the Palladio Component Model (PCM) are two architectural performance modeling formalisms applied in this context. This paper compares DML to PCM concerning similarities, differences and semantic gaps. Based on this, we propose a mapping from DML to PCM for which we implemented a tool realizing an automated transformation.

@inproceedings{WaEiHi2015-SPP-DML2PCM,
  abstract = {Model-based performance predictions and reconfigurations enable optimizing resource efficiency while ensuring that Quality-of-Service demands are met in today's complex ITsystems. The Descartes Modeling Language (DML) and the Palladio Component Model (PCM) are two architectural performance modeling formalisms applied in this context. This paper compares DML to PCM concerning similarities, differences and semantic gaps. Based on this, we propose a mapping from DML to PCM for which we implemented a tool realizing an automated transformation.},
  author = {Walter, J{\"u}rgen and Eismann, Simon and Hildebrandt, Adrian},
  booktitle = {Proceedings of the 2015 Symposium on Software Performance (SSP)},
  keywords = {t_workshop},
  month = 11,
  title = {{Automated Transformation of Descartes Modeling Language to Palladio Component Model}},
  year = 2015
}

%0 Conference Paper
%1 WaEiHi2015-SPP-DML2PCM
%A Walter, J{ü}rgen
%A Eismann, Simon
%A Hildebrandt, Adrian
%B Proceedings of the 2015 Symposium on Software Performance (SSP)
%D 2015
%T {Automated Transformation of Descartes Modeling Language to Palladio Component Model}
%X Model-based performance predictions and reconfigurations enable optimizing resource efficiency while ensuring that Quality-of-Service demands are met in today's complex ITsystems. The Descartes Modeling Language (DML) and the Palladio Component Model (PCM) are two architectural performance modeling formalisms applied in this context. This paper compares DML to PCM concerning similarities, differences and semantic gaps. Based on this, we propose a mapping from DML to PCM for which we implemented a tool realizing an automated transformation.

{Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study}. Huber, Nikolaus; Walter, J{ü}rgen; B{ä}hr, Manuel; Kounev, Samuel; in Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC) (2015). IEEE.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.

@inproceedings{HuWaBaKo2015-ICCAC-BlueYonder,
  abstract = {Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.},
  author = {Huber, Nikolaus and Walter, J{\"u}rgen and B{\"a}hr, Manuel and Kounev, Samuel},
  booktitle = {Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC)},
  keywords = {Self-adaptive-systems},
  month = {09},
  publisher = {IEEE},
  title = {{Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study}},
  year = 2015
}

%0 Conference Paper
%1 HuWaBaKo2015-ICCAC-BlueYonder
%A Huber, Nikolaus
%A Walter, J{ü}rgen
%A B{ä}hr, Manuel
%A Kounev, Samuel
%B Proceedings of the 2015 IEEE International Conference on Cloud and Autonomic Computing (ICCAC)
%D 2015
%I IEEE
%T {Model-based Autonomic and Performance-aware System Adaptation in Heterogeneous Resource Environments: A Case Study}
%X Recent trends like cloud computing show that service providers increasingly adopt to modern self-adaptive system architectures promising higher resource efficiency and lower operating costs. Blue Yonder is such a service provider in the field of predictive analytics, looking for novel approaches to automatically adapt their service infrastructure to changing customer workloads and service-level agreements. In this paper, we present a realistic case study in which we apply a holistic modelbased approach to build a self-adaptive system that automatically maintains performance requirements and resource efficiency in the heterogeneous resource environment of Blue Yonder. As modeling formalism, we employ the Descartes Modeling Language (DML), a domain-specific language for modeling the performance behavior and run-time adaptation processes of modern dynamic IT systems. This case study is an exemplar for applying a holistic model-based approach to engineering performance-aware system adaptation. More specifically, the results evaluate the applicability and effectiveness of our approach and demonstrate that DML provides suitable modeling abstractions that can be used as a basis for self-adaptive performance and resource management.

Performance-oriented {DevOps}: {A} Research Agenda. (SPEC-RG-2015-01), Brunnert, Andreas; van Hoorn, Andre; Willnecker, Felix; Danciu, Alexandru; Hasselbring, Wilhelm; Heger, Christoph; Herbst, Nikolas; Jamshidi, Pooyan; Jung, Reiner; von Kistowski, Joakim; Koziolek, Anne; Kross, Johannes; Spinner, Simon; V{ö}gele, Christian; Walter, J{ü}rgen; Wert, Alexander; (2015). SPEC Research Group --- DevOps Performance Working Group, Standard Performance Evaluation Corporation (SPEC).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]

DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.

@techreport{SPEC-RG-2015-001-DevOpsPerformanceResearchAgenda,
  abstract = {DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.},
  author = {Brunnert, Andreas and van Hoorn, Andre and Willnecker, Felix and Danciu, Alexandru and Hasselbring, Wilhelm and Heger, Christoph and Herbst, Nikolas and Jamshidi, Pooyan and Jung, Reiner and von Kistowski, Joakim and Koziolek, Anne and Kross, Johannes and Spinner, Simon and V{\"o}gele, Christian and Walter, J{\"u}rgen and Wert, Alexander},
  institution = {SPEC Research Group — DevOps Performance Working Group, Standard Performance Evaluation Corporation (SPEC)},
  keywords = {SPEC},
  month = {08},
  number = {SPEC-RG-2015-01},
  title = {Performance-oriented {DevOps}: {A} Research Agenda},
  year = 2015
}

%0 Report
%1 SPEC-RG-2015-001-DevOpsPerformanceResearchAgenda
%A Brunnert, Andreas
%A van Hoorn, Andre
%A Willnecker, Felix
%A Danciu, Alexandru
%A Hasselbring, Wilhelm
%A Heger, Christoph
%A Herbst, Nikolas
%A Jamshidi, Pooyan
%A Jung, Reiner
%A von Kistowski, Joakim
%A Koziolek, Anne
%A Kross, Johannes
%A Spinner, Simon
%A V{ö}gele, Christian
%A Walter, J{ü}rgen
%A Wert, Alexander
%D 2015
%N SPEC-RG-2015-01
%T Performance-oriented {DevOps}: {A} Research Agenda
%U https://research.spec.org/fileadmin/user_upload/documents/wg_devops/endorsed_publications/SPEC-RG-2015-001-DevOpsPerformanceResearchAgenda.pdf
%X DevOps is a trend towards a tighter integration between development (Dev) and operations (Ops) teams. The need for such an integration is driven by the requirement to continuously adapt enterprise applications (EAs) to changes in the business environment. As of today, DevOps concepts have been primarily introduced to ensure a constant flow of features and bug fixes into new releases from a functional perspective. In order to integrate a non-functional perspective into these DevOps concepts this report focuses on tools, activities, and processes to ensure one of the most important quality attributes of a software system, namely performance. Performance describes system properties concerning its timeliness and use of resources. Common metrics are response time, throughput, and resource utilization. Performance goals for EAs are typically defined by setting upper and/or lower bounds for these metrics and specific business transactions. In order to ensure that such performance goals can be met, several activities are required during development and operation of these systems as well as during the transition from Dev to Ops. Activities during development are typically summarized by the term Software Performance Engineering (SPE), whereas activities during operations are called Application Performance Management (APM). SPE and APM were historically tackled independently from each other, but the newly emerging DevOps concepts require and enable a tighter integration between both activity streams. This report presents existing solutions to support this integration as well as open research challenges in this area. The report starts by defining EAs and summarizes their characteristics that make performance evaluations for these systems particularly challenging. It continues by describing our understanding of DevOps and explaining the roots of this trend to set the context for the remaining parts of the report. Afterwards, performance management activities that are common in both life cycle phases are explained, until the particularities of SPE and APM are discussed in separate sections. Finally, the report concludes by outlining activities and challenges to support the rapid iteration between Dev and Ops.

{Parallel Simulation of Queueing Petri Nets}. Walter, J{ü}rgen; Spinner, Simon; Kounev, Samuel; in EAI Endorsed Transactions on Industrial Networks and Intelligent Systems (2015). 16(8) ACM.

[ BibTeX ]
[ EndNote ]

@article{WaSpKo2016-EAI-parallelSimulation,
  author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel},
  journal = {EAI Endorsed Transactions on Industrial Networks and Intelligent Systems},
  keywords = {se2},
  month = {08},
  number = 8,
  publisher = {ACM},
  title = {{Parallel Simulation of Queueing Petri Nets}},
  volume = 16,
  year = 2015
}

%0 Journal Article
%1 WaSpKo2016-EAI-parallelSimulation
%A Walter, J{ü}rgen
%A Spinner, Simon
%A Kounev, Samuel
%D 2015
%I ACM
%J EAI Endorsed Transactions on Industrial Networks and Intelligent Systems
%N 8
%T {Parallel Simulation of Queueing Petri Nets}
%V 16

{Parallel Simulation of Queueing Petri Nets}. Walter, J{ü}rgen; Spinner, Simon; Kounev, Samuel; in Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015) (2015).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

@inproceedings{WaSpKo-SIMUTools-QPNParallelSimulation,
  abstract = {Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.},
  author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel},
  booktitle = {Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015)},
  keywords = {QPN},
  month = {08},
  title = {{Parallel Simulation of Queueing Petri Nets}},
  year = 2015
}

%0 Conference Paper
%1 WaSpKo-SIMUTools-QPNParallelSimulation
%A Walter, J{ü}rgen
%A Spinner, Simon
%A Kounev, Samuel
%B Proceedings of the Eighth EAI International Conference on Simulation Tools and Techniques (SIMUTools 2015)
%D 2015
%T {Parallel Simulation of Queueing Petri Nets}
%X Queueing Petri nets show high accurracy to model and analyze cloud scenarios. To apply these models in online scenarios a speedup of simulation is desirable. One way to speedup simulation is parallelization. Decades of intensive research showed no general solution for parallel discrete event simulation, which provides reasonable speedups. We research the application of parallel simulation for cloud scenarios. We develop a parallel simulation engine that shows speedup for certain scenarios common in cloud models. We present a parallel simualtion engine that employs application level and event level parallelism. We parallelize the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

{Parallel Simulation of Queueing Petri Nets}. Walter, J{ü}rgen; Spinner, Simon; Kounev, Samuel; (2014, November).

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{talk-Walter-SOSP14,
  author = {Walter, J{\"u}rgen and Spinner, Simon and Kounev, Samuel},
  howpublished = {Technical Talk at Symposion on Software Performance (SOSP) 2014},
  keywords = {QPN},
  month = 11,
  title = {{Parallel Simulation of Queueing Petri Nets}},
  year = 2014
}

%0 Generic
%1 talk-Walter-SOSP14
%A Walter, J{ü}rgen
%A Spinner, Simon
%A Kounev, Samuel
%D 2014
%T {Parallel Simulation of Queueing Petri Nets}

{LibReDE: A Library for Resource Demand Estimation}. Spinner, Simon; Walter, J{ü}rgen; (2014, November).

[ BibTeX ]
[ EndNote ]
[ Download ]

@misc{talk-Spinner-SOSP14,
  author = {Spinner, Simon and Walter, J{\"u}rgen},
  howpublished = {Technical Talk at Symposion on Software Performance (SOSP) 2014},
  keywords = {Talks},
  month = 11,
  title = {{LibReDE: A Library for Resource Demand Estimation}},
  year = 2014
}

%0 Generic
%1 talk-Spinner-SOSP14
%A Spinner, Simon
%A Walter, J{ü}rgen
%D 2014
%T {LibReDE: A Library for Resource Demand Estimation}

{Parallel Simulation of Queueing Petri Net Models}. Walter, J{ü}rgen; (2013, October). Karlsruhe Institute of Technology (KIT), Am Fasanengarten 5, 76131 Karlsruhe, Germany.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

For years the CPU clock frequency was the key to improve processor performance. Nowadays, modern processors enable performance improvements by increasing the number of cores. However, existing software needs to be adapted to be able to utilize multiple cores. Such an adaptation poses many challenges in the field of discrete-event software simulation. Decades of intensive research have been spent to find a general solution for parallel discrete event simulation. In this context, QNs and PNs have been extensively studied. However, to the best of our knowledge, there is only one previous work that considers the concurrent simulation of QPN [Juergens1997]. This work focuses on comparing different synchronization algorithms and excludes a majority of lookahead calculation and net decomposition. In this thesis, we build upon and extend this work. For this purpose, we adapted and extended findings from QNs, PNs and parallel simulation in general. We apply our findings to SimQPN, which is a sequential simulation engine for QPN. Among other application areas, SimQPN is currently applied to online performance prediction for which a speedup due to parallelization is desirable. We present a parallel SimQPN implementation that employs application level and event level parallelism. A validation ensures the functional correctness of the new parallel implementations. The parallelization of multiple runs enables almost linear speedup. We parallelized the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

@mastersthesis{Walter2013-parallel-simulation,
  abstract = {For years the CPU clock frequency was the key to improve processor performance. Nowadays, modern processors enable performance improvements by increasing the number of cores. However, existing software needs to be adapted to be able to utilize multiple cores. Such an adaptation poses many challenges in the field of discrete-event software simulation. Decades of intensive research have been spent to find a general solution for parallel discrete event simulation. In this context, QNs and PNs have been extensively studied. However, to the best of our knowledge, there is only one previous work that considers the concurrent simulation of QPN [Juergens1997]. This work focuses on comparing different synchronization algorithms and excludes a majority of lookahead calculation and net decomposition. In this thesis, we build upon and extend this work. For this purpose, we adapted and extended findings from QNs, PNs and parallel simulation in general. We apply our findings to SimQPN, which is a sequential simulation engine for QPN. Among other application areas, SimQPN is currently applied to online performance prediction for which a speedup due to parallelization is desirable. We present a parallel SimQPN implementation that employs application level and event level parallelism. A validation ensures the functional correctness of the new parallel implementations. The parallelization of multiple runs enables almost linear speedup. We parallelized the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.},
  address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany},
  author = {Walter, J{\"u}rgen},
  keywords = {QPN},
  month = 10,
  school = {Karlsruhe Institute of Technology (KIT)},
  title = {{Parallel Simulation of Queueing Petri Net Models}},
  type = {Master Thesis},
  year = 2013
}

%0 Thesis
%1 Walter2013-parallel-simulation
%A Walter, J{ü}rgen
%C Am Fasanengarten 5, 76131 Karlsruhe, Germany
%D 2013
%T {Parallel Simulation of Queueing Petri Net Models}
%X For years the CPU clock frequency was the key to improve processor performance. Nowadays, modern processors enable performance improvements by increasing the number of cores. However, existing software needs to be adapted to be able to utilize multiple cores. Such an adaptation poses many challenges in the field of discrete-event software simulation. Decades of intensive research have been spent to find a general solution for parallel discrete event simulation. In this context, QNs and PNs have been extensively studied. However, to the best of our knowledge, there is only one previous work that considers the concurrent simulation of QPN [Juergens1997]. This work focuses on comparing different synchronization algorithms and excludes a majority of lookahead calculation and net decomposition. In this thesis, we build upon and extend this work. For this purpose, we adapted and extended findings from QNs, PNs and parallel simulation in general. We apply our findings to SimQPN, which is a sequential simulation engine for QPN. Among other application areas, SimQPN is currently applied to online performance prediction for which a speedup due to parallelization is desirable. We present a parallel SimQPN implementation that employs application level and event level parallelism. A validation ensures the functional correctness of the new parallel implementations. The parallelization of multiple runs enables almost linear speedup. We parallelized the execution of a single run by the use of a conservative barrier-based synchronization algorithm. The speedup for a single run depends on the capability of the model. Hence, a number of experiments on different net characteristics were conducted showing that for certain models a superlinear speedup is possible.

{Enhancing the Usability of Outliers by Description of Local Deviation}. Walter, J{ü}rgen; (2012, July). Karlsruhe Institute of Technology (KIT), Am Fasanengarten 5, 76131 Karlsruhe, Germany.

[ BibTeX ]
[ EndNote ]

@mastersthesis{Walter2012-outlier-description,
  address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany},
  author = {Walter, J{\"u}rgen},
  keywords = {se2},
  month = {07},
  school = {Karlsruhe Institute of Technology (KIT)},
  title = {{Enhancing the Usability of Outliers by Description of Local Deviation}},
  type = {Bachelor Thesis},
  year = 2012
}

%0 Thesis
%1 Walter2012-outlier-description
%A Walter, J{ü}rgen
%C Am Fasanengarten 5, 76131 Karlsruhe, Germany
%D 2012
%T {Enhancing the Usability of Outliers by Description of Local Deviation}

QPME 2.0 User’s Guide Kounev, Samuel; Spinner, Simon; Walter, J{ü}rgen; (2011). Karlsruhe Institute of Technology, Am Fasanengarten 5, 76131 Karlsruhe, Germany.

[ BibTeX ]
[ EndNote ]

@manual{KoSp2011-QPME20-UserGuide,
  address = {Am Fasanengarten 5, 76131 Karlsruhe, Germany},
  author = {Kounev, Samuel and Spinner, Simon and Walter, J{\"u}rgen},
  keywords = {QPN},
  month = {05},
  organization = {Karlsruhe Institute of Technology},
  title = {QPME 2.0 User's Guide},
  year = 2011
}

%0 Generic
%1 KoSp2011-QPME20-UserGuide
%A Kounev, Samuel
%A Spinner, Simon
%A Walter, J{ü}rgen
%C Am Fasanengarten 5, 76131 Karlsruhe, Germany
%D 2011
%T QPME 2.0 User's Guide

Publications

Data privacy protection

Data privacy protection

Picture credits