Dissertation oder Abschlussarbeit
[
nach oben
]
Artikel in einem Journal, einer Zeitung oder einem Magazin
[
nach oben
]
Artikel in Konferenzband
[
nach oben
]
-
To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows. Züfle, Marwin; Krupitzer, Christian; Erhard, Florian; Grohmann, Johannes; Kounev, Samuel; in MMB 2020 (2020).
Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.
-
Predicting the Costs of Serverless Workflows. Eismann, Simon; Grohmann, Johannes; van Eyk, Erwin; Herbst, Nikolas; Kounev, Samuel; in ICPE'20 (2020).
Acceptance Rate: 23.4% (15/64)
Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.
-
An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series. Bauer, André; Züfle, Marwin; Grohmann, Johannes; Schmitt, Norbert; Herbst, Nikolas; Kounev, Samuel; (2020).
-
Model-based Performance Predictions for SDN-based Networks: A Case Study. Herrnleben, Stefan; Rygielski, Pitor; Grohmann, Johannes; Eismann, Simon; Hossfeld, Tobias; Kounev, Samuel; in MMB 2020 (2020).
Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.
-
Monitorless: Predicting Performance Degradation in Cloud Applications with Machine Learning. Grohmann, Johannes; Nicholson, Patrick K.; Iglesias, Jesus Omana; Kounev, Samuel; Lugones, Diego; in Middleware 2019 (2019). 149--162.
-
Detecting Parametric Dependencies for Performance Models Using Feature Selection Techniques. Grohmann, Johannes; Eismann, Simon; Elflein, Sven; Mazkatli, Manar; von Kistowski, Jóakim; Kounev, Samuel; in MASCOTS '19 (2019).
Acceptance Rate: 23.8% (29/122)
Architectural performance models are a common approach to predict the performance properties of a software system. Parametric dependencies, which describe the relation between the input parameters of a component and its performance properties, significantly increase the prediction accuracy of architectural performance models. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which are often infeasible. In this paper, we introduce an approach to automatically identify parametric dependencies from monitoring data using feature selection techniques from the area of machine learning. We evaluate the applicability of three techniques selected from each of the three groups of feature selection methods: a filter method, an embedded method, and a wrapper method. Our evaluation shows that the filter technique outperforms the other approaches. Based on these results, we apply this technique to a distributed micro-service web-shop, where it correctly identifies 11 performance-relevant dependencies, achieving a precision of 91.7% based on a manually labeled gold-standard.
-
Utilizing Clustering to Optimize Resource Demand Estimation Approaches. Grohmann, Johannes; Eismann, Simon; Bauer, Andre; Zuefle, Marwin; Herbst, Nikolas; Kounev, Samuel; (2019). 134-139.
Resource demands are crucial parameters for modeling and predicting the performance of software systems. Direct measurement of these resource demands is usually infeasible due to instrumentation overheads causing measurement interferences and perturbation in production environments. Thus, a number of statistical estimation approaches (e.g., based on optimization, regression or Kalman filters) have been proposed in the literature. Most of these approaches are parameterized. These parameters influence the estimation quality and the required computation time. Existing work uses historical data as training sets to optimize those parameters and to minimize the estimation error of those approaches. However, if the data traces are fundamentally different, the optimal parameter settings are different as well. In this paper, we propose to use automated clustering in order to group training sets into groups of similar optimization behavior. This way, optimization can be specifically tailored to certain groups of traces in a self-aware manner. During run-time, every trace is first sorted into a cluster, where the respective cluster-wide parameter optimum can be applied. A preliminary case study shows that clustering can provide promising improvements.
-
On Learning Parametric Dependencies from Monitoring Data. Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; (2019).
-
Predicting Server Power Consumption from Standard Rating Results. von Kistowski, Jóakim; Grohmann, Johannes; Schmitt, Norbert; Kounev, Samuel; in ICPE '19 (2019).
Full Paper Acceptance Rate: 18.6% (13/70)
Data center providers and server operators try to reduce the power consumption of their servers. Finding an energy efficient server for a specific target application is a first step in this regard. Estimating the power consumption of an application on an unavailable server is difficult, as nameplate power values are generally overestimations. Offline power models are able to predict the consumption accurately, but are usually intended for system design, requiring very specific and detailed knowledge about the system under consideration. In this paper, we introduce an offline power prediction method that uses the results of standard power rating tools. The method predicts the power consumption of a specific application for multiple load levels on a target server that is otherwise unavailable for testing. We evaluate our approach by predicting the power consumption of three applications on different physical servers. Our method is able to achieve an average prediction error of 9.49% for three workloads running on real-world, physical servers.
-
On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework. D'Angelo, M.; Gerasimou, S.; Ghahremani, S.; Grohmann, J.; Nunes, I.; Pournaras, E.; Tomforde, S.; (2019). 13-24.
-
Integrating Statistical Response Time Models in Architectural Performance Models. Eismann, Simon; Grohmann, Johannes; Walter, Jürgen; von Kistowski, Jóakim; Kounev, Samuel; (2019). 71-80.
Acceptance Rate: 21,9% (21/96)
Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.
-
Systematic Search for Optimal Resource Configurations of Distributed Applications. Bauer, André; Eismann, Simon; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; (2019).
-
How is Performance Addressed in DevOps? Bezemer, Cor-Paul; Eismann, Simon; Ferme, Vincenzo; Grohmann, Johannes; Heinrich, Robert; Jamshidi, Pooyan; Shang, Weiyi; van Hoorn, André; Villavicencio, Mónica; Walter, Jürgen; Willnecker, Felix; (2019). 45--50.
-
The Vision of Self-Aware Performance Models. Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; (2018). 60-63.
Performance models are necessary components of self-aware computing systems, as they allow such systems to reason about their own state and behavior. Research in this field has developed a multitude of approaches to create, maintain, and solve performance models. In this paper, we propose a meta-self-aware computing approach making the processes of model creation, maintenance and solution themselves self-aware. This enables the automated selection and adaption of software performance engineering approaches specifically tailored to the system under study.
-
TeaStore: A Micro-Service Reference Application for Benchmarking, Modeling and Resource Management Research. von Kistowski, Jóakim; Eismann, Simon; Schmitt, Norbert; Bauer, André; Grohmann, Johannes; Kounev, Samuel; in MASCOTS '18 (2018). 223--236.
Acceptance Rate: 29.5% (23/78)
Modern distributed applications offer complex performance behavior and many degrees of freedom regarding deployment and configuration. Researchers employ various methods of analysis, modeling, and management that leverage these degrees of freedom to predict or improve non-functional properties of the software under consideration. In order to demonstrate and evaluate their applicability in the real world, methods resulting from such research areas require test and reference applications that offer a range of different behaviors, as well as the necessary degrees of freedom. Existing production software is often inaccessible for researchers or closed off to instrumentation. Existing testing and benchmarking frameworks, on the other hand, are either designed for specific testing scenarios, or they do not offer the necessary degrees of freedom. Further, most test applications are difficult to deploy and run, or are outdated. In this paper, we introduce the TeaStore, a state-of-the-art micro-service-based test and reference application. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. The TeaStore allows evaluating performance modeling and resource management techniques; it also offers instrumented variants to enable extensive run-time analysis. We demonstrate TeaStore's use in three contexts: performance modeling, cloud resource management, and energy efficiency analysis. Our experiments show that TeaStore can be used for evaluating novel approaches in these contexts and also motivates further research in the areas of performance modeling and resource management.
-
A SPEC RG Cloud Group's Vision on the Performance Challenges of FaaS Cloud Architectures. van Eyk, Erwin; Iosup, Alexandru; Abad, Cristina L.; Grohmann, Johannes; Eismann, Simon; in ICPE '18 (2018). 21--24.
As a key part of the serverless computing paradigm, Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues. However, there are several performance-related issues surrounding the state-of-the-art FaaS platforms that can deter widespread adoption of FaaS, including sizeable overheads, unreliable performance, and new forms of the cost-performance trade-off. In this work we, the SPEC RG Cloud Group, identify six performance-related challenges that arise specifically in this FaaS model, and present our roadmap to tackle these problems in the near future. This paper aims at motivating the community to solve these challenges together.
-
On the Value of Service Demand Estimation for Auto-Scaling. Bauer, André; Grohmann, Johannes; Herbst, Nikolas; Kounev, Samuel; (2018).
-
Tools for Declarative Performance Engineering. Walter, Jürgen; Eismann, Simon; Grohmann, Johannes; Okanovic, Dusan; Kounev, Samuel; in ICPE '18 (2018). 53--56.
Performance is of particular relevance to software system design, operation, and evolution. However, the application of performance engineering approaches to solve a given user concern is challenging and requires expert knowledge. In this tutorial paper, we guide the reader step-by-step through the answering of performance concerns following the idea of declarative performance engineering. We explain tools available online, which can be used for automating huge parts of the software performance engineering process. In particular, we present a performance concern language, for which we provide automated answering and visualization referring to measurement-based and model-based analysis. We also detail how to derive performance models using automated extraction of architectural performance models and modeling of parametric dependencies.
-
TeaStore: A Micro-Service Reference Application for Cloud Researchers. Eismann, Simon; v. Kistowski, Jóakim; Grohmann, Johannes; Bauer, André; Schmitt, Norbert; Herbst, Nikolas; Kounev, Samuel; (2018). 11-12.
Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed cloud applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. Further, most cloud reference applications are difficult to deploy and run. In this paper, we present the TeaStore, a micro-service-based test and reference cloud application. TeaStore offers services with various performance characteristics and a high degree of freedom regarding its deployment and configuration to be used as a cloud reference application for researchers. The TeaStore is designed for the evaluation of performance modeling and resource management techniques. We invite cloud researchers to use the TeaStore and provide it open-source, extendable, easily deployable and monitorable.
-
Black-box Learning of Parametric Dependencies for Performance Models. Ackermann, Vanessa; Grohmann, Johannes; Eismann, Simon; Kounev, Samuel; in CEUR Workshop Proceedings (2018).
Modeling parametric dependencies in architectural performance models increases performance prediction accuracy. However, manually modeling parametric dependencies is time-intensive and requires expert knowledge. Existing automated extraction approaches require dedicated performance tests, which is often infeasible. In this paper, we propose to characterize parametric dependencies based on monitoring data. We create a representative dataset and show that different machine learning approaches perform best, depending on the characteristics of the dependency. Based on these results, we introduce a meta-selector that chooses the most suitable machine learning approach based on the dependency characteristics. In our evaluation, the meta-selector reduces the prediction error compared to the best individual machine learning approach, SVR, by 30%. As a proof of concept, we show that our approach is capable of automatically characterizing a manually modeled dependency from a previous case-study, resulting in a response time prediction accuracy of 92.8%.
-
Using Machine Learning for Recommending Service Demand Estimation Approaches. Grohmann, Johannes; Herbst, Nikolas; Spinner, Simon; Kounev, Samuel; (2018). 473-480.
-
Self-Tuning Resource Demand Estimation. Grohmann, Johannes; Herbst, Nikolas; Spinner, Simon; Kounev, Samuel; (2017).
Automatic Identification of Parametric Dependencies for Performance Models. Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2018).Würzburg Software Engineering Award sponsored by Bosch Rexroth
A Taxonomy of Techniques for SLO Failure Prediction in Software Systems. in Computers (2020). 9(1) 10.
Model-based Performance Predictions for SDN-based Networks: A Case Study. in MMB 2020 (2020).
