PMX

Performance Model eXtractor

The manual creation of architectural performance models is very complex, time intense and error prone. The Performance Model eXtractor (PMX) tool automates the extraction of architectural performance models form measurement data. Currently, PMX supports logs of the Kieker Monitoring Framework as input data format. PMX separates the learning of generic aspects from model creation and is able to extract models of different formalisms. Currently there are builder implementations for Palladio Component Model and Descartes Modeling Language. More information can be found on the following pages:

Webservices for PCM and DML (new)
Download (eclipse update site and standalone archive)
source code
jenkins (currently only accessible within network of the university of wuerzburg)
License

If you have any questions, please contact Jürgen Walter.

Mailing List

To stay updated on our tools, please subscribe to our descartes-tools mailing list (low traffic, only announcements related to our tools)

Publications

2019[ to top ]

Automation in Software Performance Engineering Based on a Declarative Specification of Concerns J. C. Walter; Thesis; Universität Würzburg. (2019).
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@phdthesis{Walter2018-phd, author = {Walter, J{\"u}rgen Christian}, keywords = {PMX}, school = {Universit{\"a}t W{\"u}rzburg}, title = {Automation in Software Performance Engineering Based on a Declarative Specification of Concerns}, type = {doctoralthesis}, year = 2019 }

%0 Thesis %1 Walter2018-phd %A Walter, Jürgen Christian %D 2019 %T Automation in Software Performance Engineering Based on a Declarative Specification of Concerns %U https://opus.bibliothek.uni-wuerzburg.de/frontdoor/index/index/docId/18090

2018[ to top ]

Tools for Declarative Performance Engineering J. Walter; S. Eismann; J. Grohmann; D. Okanovic; S. Kounev; in Companion of the 2018 ACM/SPEC International Conference on Performance Engineering (2018). 53–56.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
- [ DOI ]
Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.

@inproceedings{WaEiGrOkKo2018-ICPE-Tools-for-DPE-Tutorial, abstract = {Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.}, address = {New York, NY, USA}, author = {Walter, J{\"u}rgen and Eismann, Simon and Grohmann, Johannes and Okanovic, Dusan and Kounev, Samuel}, booktitle = {Companion of the 2018 ACM/SPEC International Conference on Performance Engineering}, keywords = {DQL}, pages = {53–56}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '18}, title = {{Tools for Declarative Performance Engineering}}, year = 2018 }

%0 Conference Paper %1 WaEiGrOkKo2018-ICPE-Tools-for-DPE-Tutorial %A Walter, Jürgen %A Eismann, Simon %A Grohmann, Johannes %A Okanovic, Dusan %A Kounev, Samuel %B Companion of the 2018 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2018 %I Association for Computing Machinery (ACM) %P 53--56 %R 10.1145/3185768.3185777 %T Tools for Declarative Performance Engineering %U https://doi.org/10.1145/3185768.3185777 %X Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies. %@ 9781450356299

2017[ to top ]

Providing Model-Extraction-as-a-Service for Architectural Performance Models J. Walter; S. Eismann; N. Reed; S. Kounev; in Proceedings of the 2017 Symposium on Software Performance (SSP) (2017).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.

@inproceedings{WaEiReKo2017-SPP-MEaaS, abstract = {Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.}, author = {Walter, J{\"u}rgen and Eismann, Simon and Reed, Nikolai and Kounev, Samuel}, booktitle = {Proceedings of the 2017 Symposium on Software Performance (SSP)}, keywords = {t_workshop}, month = 11, title = {{Providing Model-Extraction-as-a-Service for Architectural Performance Models}}, year = 2017 }

%0 Conference Paper %1 WaEiReKo2017-SPP-MEaaS %A Walter, Jürgen %A Eismann, Simon %A Reed, Nikolai %A Kounev, Samuel %B Proceedings of the 2017 Symposium on Software Performance (SSP) %D 2017 %T Providing Model-Extraction-as-a-Service for Architectural Performance Models %X Architectural performance models can be leveraged to explore performance properties of software systems during design-time and run-time. We see a reluctance from industry to adopt model-based analysis approaches due to the required expertise and modeling effort. Building models from scratch in an editor does not scale for medium and large-scale systems in an industrial context. Existing open-source performance model extraction approaches imply signi cant initial efforts which might be challenging for layman users. To simplify usage, we provide the extraction of architectural performance models based on application monitoring traces as a web service. Model-Extraction-as-a-Service (MEaaS) solves the usability problem and lowers the initial effort of applying model-based analysis approaches.
CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches T. F. Düllmann; R. Heinrich; A. van Hoorn; T. Pitakrat; J. Walter; F. Willnecker; in Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017) (2017).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.

@inproceedings{DuHeHoPiWaWi2017-CASPA, abstract = {Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.}, author = {D{\"u}llmann, Thomas F. and Heinrich, Robert and van Hoorn, Andre and Pitakrat, Teerat and Walter, J{\"u}rgen and Willnecker, Felix}, booktitle = {Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017)}, keywords = {PMX}, publisher = {IEEE}, title = {CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches}, year = 2017 }

%0 Conference Paper %1 DuHeHoPiWaWi2017-CASPA %A Düllmann, Thomas F. %A Heinrich, Robert %A van Hoorn, Andre %A Pitakrat, Teerat %A Walter, Jürgen %A Willnecker, Felix %B Proceedings of the 2017 IEEE International Conference on Software Architecture (ICSA 2017) %D 2017 %I IEEE %T CASPA: A Platform for Comparability of Architecture-based Software Performance Engineering Approaches %U http://eprints.uni-kiel.de/37422/ %X Setting up an experimental evaluation for architecture-based Software Performance Engineering (SPE) approaches requires enormous efforts. This includes the selection and installation of representative applications, usage profiles, supporting tools, infrastructures, etc. Quantitative comparisons with related approaches are hardly possible due to limited repeatability of previous experiments by other researchers. This paper presents CASPA, a ready-to-use and extensible evaluation platform that already includes example applications and state-of-the-art SPE components, such as monitoring and model extraction. The platform explicitly provides interfaces to replace applications and components by custom(ized) components. The platform builds on state-of-the-art technologies such as container-based virtualization.
An Expandable Extraction Framework for Architectural Performance Models J. Walter; C. Stier; H. Koziolek; S. Kounev; in Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS’17) (2017).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.

@inproceedings{WaStKoKo2017-QUDOS-PMXBuilder, abstract = {Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.}, author = {Walter, J{\"u}rgen and Stier, Christian and Koziolek, Heiko and Kounev, Samuel}, booktitle = {{Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17)}}, keywords = {t_workshop}, month = {April}, publisher = {ACM}, title = {{An Expandable Extraction Framework for Architectural Performance Models}}, year = 2017 }

%0 Conference Paper %1 WaStKoKo2017-QUDOS-PMXBuilder %A Walter, Jürgen %A Stier, Christian %A Koziolek, Heiko %A Kounev, Samuel %B Proceedings of the 3rd International Workshop on Quality-Aware DevOps (QUDOS'17) %D 2017 %I ACM %T An Expandable Extraction Framework for Architectural Performance Models %X Providing users with Quality of Service (QoS) guarantees and the prevention of performance problems are challenging tasks for software systems. Architectural performance models can be applied to explore performance properties of a software system at design time and run time. At design time, architectural performance models support reasoning on effects of design decisions. At run time, they enable automatic reconfigurations by reasoning on the effects of changing user behavior. In this paper, we present a framework for the extraction of architectural performance models based on monitoring log files generalizing over the targeted architectural modeling language. Using the presented framework, the creation of a performance model extraction tool for a specific modeling formalism requires only the implementation of a key set of object creation routines specific to the formalism. Our framework integrates them with extraction techniques that apply to many architectural performance models, e.g., resource demand estimation techniques. This lowers the effort to implement performance model extraction tools tremendously through a high level of reuse. We evaluate our framework presenting builders for the Descartes Modeling Language (DML) and the Palladio Component Model (PCM). For the extracted models we compare simulation results with measurements receiving accurate results.
Mapping of Service Level Objectives to Performance Queries J. Walter; D. Okanovic; S. Kounev; in Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C’17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) (2017).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.

@inproceedings{WaOkKo2017-WOSPC-SLO2Queries, abstract = {The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.}, author = {Walter, J{\"u}rgen and Okanovic, Dusan and Kounev, Samuel}, booktitle = {{Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017)}}, keywords = {DQL}, month = {April}, publisher = {ACM}, title = {{Mapping of Service Level Objectives to Performance Queries}}, year = 2017 }

%0 Conference Paper %1 WaOkKo2017-WOSPC-SLO2Queries %A Walter, Jürgen %A Okanovic, Dusan %A Kounev, Samuel %B Proceedings of the 2017 Workshop on Challenges in Performance Methods for Software Development (WOSP-C'17) co-located with 8th ACM/SPEC International Conference on Performance Engineering (ICPE 2017) %D 2017 %I ACM %T Mapping of Service Level Objectives to Performance Queries %X The concept of service level agreements (SLAs) defines the idea of a reliable contract between service providers and their users. SLAs provide information on the scope, the quality and the responsibilities of a service and its provider. Service level objectives (SLOs) define the detailed, measurable conditions of the SLAs. After service deployment, SLAs are monitored, to assess potentially dangerous situations, that further lead to violation of the SLAs. However, the SLA monitoring infrastructure is usually specific to the underlying system infrastructure, lacks generalization, and is often limited to measurement-based approaches. This makes it hard to apply the results from SLA monitoring in other stages of the software life-cycle. In this paper we propose the mapping of concerns defined in SLAs to the performance metrics queries using the Descartes Query Language (DQL). The benefit of our approach is that the same performance query can then be reused for evaluation of performance concerns throughout the entire life-cycle, and regardless of which approach is used for evaluation.
Online Learning of Run-time Models for Performance and Resource Management in Data Centers J. Walter; A. D. Marco; S. Spinner; P. Inverardi; S. Kounev; in Self-Aware Computing Systems, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (Hrsg.) (2017).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.

@incollection{WaMaSpInKo-learning, abstract = {In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.}, address = {{Berlin Heidelberg, Germany}}, author = {Walter, J{\"u}rgen and Marco, Antinisca Di and Spinner, Simon and Inverardi, Paola and Kounev, Samuel}, booktitle = {{Self-Aware Computing Systems}}, editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun}, keywords = {Self-adaptive-systems}, publisher = {{Springer Verlag}}, title = {{Online Learning of Run-time Models for Performance and Resource Management in Data Centers}}, year = 2017 }

%0 Book Section %1 WaMaSpInKo-learning %A Walter, Jürgen %A Marco, Antinisca Di %A Spinner, Simon %A Inverardi, Paola %A Kounev, Samuel %B Self-Aware Computing Systems %C Berlin Heidelberg, Germany %D 2017 %E Kounev, Samuel %E Kephart, Jeffrey O. %E Milenkoski, Aleksandar %E Zhu, Xiaoyun %I Springer Verlag %T Online Learning of Run-time Models for Performance and Resource Management in Data Centers %X In this chapter, we explain how to extract and learn run-time models that a system can use for self-aware performance and resource management in data centers. We abstract from concrete formalisms and identify extraction aspects relevant for performance models. We categorize the learning aspects into: i) model structure, ii) model parametrization (estimation and calibration of model parameters), and iii) model adaptation options (change point detection and run-time reconfiguration). The chapter identifies alternative approaches for the respective model aspects. The type and granularity of each aspect depends on the characteristic of the concrete performance models.

2016[ to top ]

Automation and Simplification Through Declarative Performance Engineering J. Walter; (2016, Juni).
- [ BibTeX ]
- [ EndNote ]
@misc{Talk-Kolloquium-2016-06-14, address = {W{\"u}rzburg, Germany}, author = {Walter, J{\"u}rgen}, howpublished = {Doctoral Colloquium W{\"u}rzburg, June 14, 2016}, keywords = {Optimization}, month = {June}, title = {{Automation and Simplification Through Declarative Performance Engineering}}, year = 2016 }

%0 Generic %1 Talk-Kolloquium-2016-06-14 %A Walter, Jürgen %C Würzburg, Germany %D 2016 %T Automation and Simplification Through Declarative Performance Engineering

2015[ to top ]

Leistung von IT-Systemen vorhersagen R. Emmerich; (2015, Dezember). 1–2.
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
@misc{PMX2015, author = {Emmerich, Robert}, journal = {einBLICK}, keywords = {Media-coverage}, month = {December}, pages = {1-2}, title = {{Leistung von IT-Systemen vorhersagen}}, year = 2015 }

%0 Generic %1 PMX2015 %A Emmerich, Robert %D 2015 %J einBLICK %P 1-2 %T Leistung von IT-Systemen vorhersagen %U http://www.presse.uni-wuerzburg.de/einblick/single_special/artikel/leistung-von-it-systemen-vorhersagen/

Performance Model eXtractor

Mailing List

Publications

Sponsors

Data privacy protection

Data privacy protection

Picture credits

Your E-mail address:
Your Name (optional):