Publications

2022 [ nach oben ]

Contact Discovery in Mobile Messengers: Low-cost Attacks, Quantitative Analyses, and Efficient Mitigations. C. Hagen; C. Weinert; C. Sendner; A. Dmitrienko; T. Schneider; in ACM Transactions on Privacy and Security (2022).
(to appear)
[ BibTeX ]
 
@article{hagen2022contact, author = {Hagen, Christoph and Weinert, Christian and Sendner, Christoph and Dmitrienko, Alexandra and Schneider, Thomas}, journal = {ACM Transactions on Privacy and Security}, keywords = {international-conference-workshop-papers-book-chapters sss-group}, note = {(to appear)}, title = {Contact Discovery in Mobile Messengers: Low-cost Attacks, Quantitative Analyses, and Efficient Mitigations}, year = 2022 }

Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity. M. Leznik; J. Grohmann; N. Kliche; A. Bauer; D. Seybold; S. Eismann; S. Kounev; J. Domaschka; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 89–96.
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.
@inproceedings{leznik2022dissimilar, abstract = {Benchmarking is a core element in the toolbox of most systems researchers and is used for analyzing, comparing, and validating complex systems. In the quest for reliable benchmark results, a consensus has formed that a significant experiment must be based on multiple runs. To interpret these runs, mean and standard deviation are often used. In case of experiments where each run produces a time series, applying and comparing the mean is not easily applicable and not necessarily statistically sound. Such an approach ignores the possibility of significant differences between runs with a similar average. In order to verify this hypothesis, we conducted a survey of 1,112 publications of selected performance engineering and systems conferences canvassing open data sets from performance experiments. The identified 3 data sets purely rely on average and standard deviation. Therefore, we propose a novel analysis approach based on similarity analysis to enhance the reliability of performance evaluations. Our approach evaluates 12 (dis-)similarity measures with respect to their applicability in analysing performance measurements and identifies four suitable similarity measures. We validate our approach by demonstrating the increase in reliability for the data sets found in the survey.}, address = {New York, NY, USA}, author = {Leznik, Mark and Grohmann, Johannes and Kliche, Nina and Bauer, Andr{é} and Seybold, Daniel and Eismann, Simon and Kounev, Samuel and Domaschka, J{ö}rg}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {timeseries t_short descartes metrics analysis}, pages = {89–96}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Same, Same, but Dissimilar: Exploring Measurements for Workload Time-Series Similarity}, year = 2022 }

FedCRI: Federated Mobile Cyber-Risk Intelligence. H. Fereidooni; A. Dmitrienko; F. Madlener; P. Rieger; M. Miettinen; A.-R. Sadeghi; in The Network and Distributed System Security Symposium (NDSS) (2022).
(to appear)
[ BibTeX ]
 
@inproceedings{fereidooni2022fedcri, author = {Fereidooni, Hossein and Dmitrienko, Alexandra and Madlener, Felix and Rieger, Phillip and Miettinen, Markus and Sadeghi, Ahmad-Reza}, booktitle = {The Network and Distributed System Security Symposium (NDSS)}, keywords = {international-conference-workshop-papers-book-chapters sss-group mobile security}, note = {(to appear)}, title = {FedCRI: Federated Mobile Cyber-Risk Intelligence}, year = 2022 }

Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes. T. Prantl; T. Zeck; L. Iffländer; L. Beierlieb; A. Dmitrenko; C. Krupitzer; S. Kounev; in 2022 5th Conference on Cloud and Internet of Things (CIoT) (2022).
[ BibTeX ]
 
@inproceedings{PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark, author = {Prantl, Thomas and Zeck, Timo and Iffländer, Lukas and Beierlieb, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2022 5th Conference on Cloud and Internet of Things (CIoT)}, keywords = {t_full descartes}, organization = {IEEE}, title = {Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes}, year = 2022 }

SCRAPS: Scalable Collective Remote Attestation for Pub-Sub IoT Networks with Untrusted Proxy Verifier L. Petzi; A. E. Ben Yahya; A. Dmitrienko; G. Tsudik; T. Prantl; S. Kounev; (2022). (Vol. USENIX Security ’22)
Coming Soon
[ Abstract ] [ BibTeX ]
 
Remote Attestation (RA) is a basic security mechanism that detects malicious presence on various types of computing components, e.g., IoT devices. In a typical IoT setting, RA involves a trusted Verifier that sends a challenge to an untrusted remote Prover, which must in turn reply with a fresh and authentic evidence of being in a trustworthy state. However,most current RA schemes assume a central Verifier, which rep-resents a single point of failure. This feature is problematic when mutually suspicious stakeholders are involved. Furthermore, scalability issues arise as the number of IoT devices (Provers) grows. Although some RA schemes allow peer Provers to act as Verifiers, they involve unrealistic (for IoT devices) requirements, such as time synchronization and synchronous communication. Moreover, they incur heavy memory, computation,and communication burdens, while not considering sleeping or otherwise disconnected devices. Motivated by the need to address these limitations, we construct Scalable Collective Remote Attestation for Pub-Sub (SCRAPS), a novel collective RA scheme. It achieves scalability by outsourcing Verifier duties to a smart contract and mitigates DoS attacks against both Provers and Verifiers. It also removes the need for synchronous communication. Furthermore,RA evidence in SCRAPS is publicly verifiable, which significantly reduces the number of attestation evidence computations, thus lowering Prover burden. We report on a prototype of SCRAPS over Hyperledger Sawtooth (a blockchain geared for IoT use-cases) and evaluate its performance, scalability, and security aspects.
@proceedings{petzi2022scraps, abstract = {Remote Attestation (RA) is a basic security mechanism that detects malicious presence on various types of computing components, e.g., IoT devices. In a typical IoT setting, RA involves a trusted Verifier that sends a challenge to an untrusted remote Prover, which must in turn reply with a fresh and authentic evidence of being in a trustworthy state. However,most current RA schemes assume a central Verifier, which rep-resents a single point of failure. This feature is problematic when mutually suspicious stakeholders are involved. Furthermore, scalability issues arise as the number of IoT devices (Provers) grows. Although some RA schemes allow peer Provers to act as Verifiers, they involve unrealistic (for IoT devices) requirements, such as time synchronization and synchronous communication. Moreover, they incur heavy memory, computation,and communication burdens, while not considering sleeping or otherwise disconnected devices. Motivated by the need to address these limitations, we construct Scalable Collective Remote Attestation for Pub-Sub (SCRAPS), a novel collective RA scheme. It achieves scalability by outsourcing Verifier duties to a smart contract and mitigates DoS attacks against both Provers and Verifiers. It also removes the need for synchronous communication. Furthermore,RA evidence in SCRAPS is publicly verifiable, which significantly reduces the number of attestation evidence computations, thus lowering Prover burden. We report on a prototype of SCRAPS over Hyperledger Sawtooth (a blockchain geared for IoT use-cases) and evaluate its performance, scalability, and security aspects.}, author = {Petzi, Lukas and Ben Yahya, Ala Eddine and Dmitrienko, Alexandra and Tsudik, Gene and Prantl, Thomas and Kounev, Samuel}, keywords = {international-conference-workshop-papers-book-chapters sss-group}, note = {Coming Soon}, title = {SCRAPS: Scalable Collective Remote Attestation for Pub-Sub IoT Networks with Untrusted Proxy Verifier}, volume = {USENIX Security '22}, year = 2022 }

Model Learning for Performance Prediction of Cloud-native Microservice Applications. J. Grohmann; (2022, March).
Distinction
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
One consequence of the recent coronavirus pandemic is increased demand and use of online services around the globe. At the same time, performance requirements for modern technologies are becoming more stringent as users become accustomed to higher standards. These increased performance and availability requirements, coupled with the unpredictable usage growth, are driving an increasing proportion of applications to run on public cloud platforms as they promise better scalability and reliability. With data centers already responsible for about one percent of the world's power consumption, optimizing resource usage is of paramount importance. Simultaneously, meeting the increasing and changing resource and performance requirements is only possible by optimizing resource management without introducing additional overhead. This requires the research and development of new modeling approaches to understand the behavior of running applications with minimal information. However, the emergence of modern software paradigms makes it increasingly difficult to derive such models and renders previous performance modeling techniques infeasible. Modern cloud applications are often deployed as a collection of fine-grained and interconnected components called microservices. Microservice architectures offer massive benefits but also have broad implications for the performance characteristics of the respective systems. In addition, the microservices paradigm is typically paired with a DevOps culture, resulting in frequent application and deployment changes. Such applications are often referred to as cloud-native applications. In summary, the increasing use of ever-changing cloud-hosted microservice applications introduces a number of unique challenges for modeling the performance of modern applications. These include the amount, type, and structure of monitoring data, frequent behavioral changes, or infrastructure variabilities. This violates common assumptions of the state of the art and opens a research gap for our work. In this thesis, we present five techniques for automated learning of performance models for cloud-native software systems. We achieve this by combining machine learning with traditional performance modeling techniques. Unlike previous work, our focus is on cloud-hosted and continuously evolving microservice architectures, so-called cloud-native applications. Therefore, our contributions aim to solve the above challenges to deliver automated performance models with minimal computational overhead and no manual intervention. Depending on the cloud computing model, privacy agreements, or monitoring capabilities of each platform, we identify different scenarios where performance modeling, prediction, and optimization techniques can provide great benefits. Specifically, the contributions of this thesis are as follows: Monitorless: Application-agnostic prediction of performance degradations. To manage application performance with only platform-level monitoring, we propose Monitorless, the first truly application-independent approach to detecting performance degradation. We use machine learning to bridge the gap between platform-level monitoring and application-specific measurements, eliminating the need for application-level monitoring. Monitorless creates a single and holistic resource saturation model that can be used for heterogeneous and untrained applications. Results show that Monitorless infers resource-based performance degradation with 97% accuracy. Moreover, it can achieve similar performance to typical autoscaling solutions, despite using less monitoring information. SuanMing: Predicting performance degradation using tracing. We introduce SuanMing to mitigate performance issues before they impact the user experience. This contribution is applied in scenarios where tracing tools enable application-level monitoring. SuanMing predicts explainable causes of expected performance degradations and prevents performance degradations before they occur. Evaluation results show that SuanMing can predict and pinpoint future performance degradations with an accuracy of over 90%. SARDE: Continuous and autonomous estimation of resource demands. We present SARDE to learn application models for highly variable application deployments. This contribution focuses on the continuous estimation of application resource demands, a key parameter of performance models. SARDE represents an autonomous ensemble estimation technique. It dynamically and continuously optimizes, selects, and executes an ensemble of approaches to estimate resource demands in response to changes in the application or its environment. Through continuous online adaptation, SARDE efficiently achieves an average resource demand estimation error of 15.96% in our evaluation. DepIC: Learning parametric dependencies from monitoring data. DepIC utilizes feature selection techniques in combination with an ensemble regression approach to automatically identify and characterize parametric dependencies. Although parametric dependencies can massively improve the accuracy of performance models, DepIC is the first approach to automatically learn such parametric dependencies from passive monitoring data streams. Our evaluation shows that DepIC achieves 91.7% precision in identifying dependencies and reduces the characterization prediction error by 30% compared to the best individual approach. Baloo: Modeling the configuration space of databases. To study the impact of different configurations within distributed DBMSs, we introduce Baloo. Our last contribution models the configuration space of databases considering measurement variabilities in the cloud. More specifically, Baloo dynamically estimates the required benchmarking measurements and automatically builds a configuration space model of a given DBMS. Our evaluation of Baloo on a dataset consisting of 900 configuration points shows that the framework achieves a prediction error of less than 11% while saving up to 80% of the measurement effort. Although the contributions themselves are orthogonally aligned, taken together they provide a holistic approach to performance management of modern cloud-native microservice applications. Our contributions are a significant step forward as they specifically target novel and cloud-native software development and operation paradigms, surpassing the capabilities and limitations of previous approaches. In addition, the research presented in this paper also has a significant impact on the industry, as the contributions were developed in collaboration with research teams from Nokia Bell Labs, Huawei, and Google. Overall, our solutions open up new possibilities for managing and optimizing cloud applications and improve cost and energy efficiency.
@phdthesis{Gr2022-Dissertation, abstract = {One consequence of the recent coronavirus pandemic is increased demand and use of online services around the globe. At the same time, performance requirements for modern technologies are becoming more stringent as users become accustomed to higher standards. These increased performance and availability requirements, coupled with the unpredictable usage growth, are driving an increasing proportion of applications to run on public cloud platforms as they promise better scalability and reliability. With data centers already responsible for about one percent of the world's power consumption, optimizing resource usage is of paramount importance. Simultaneously, meeting the increasing and changing resource and performance requirements is only possible by optimizing resource management without introducing additional overhead. This requires the research and development of new modeling approaches to understand the behavior of running applications with minimal information. However, the emergence of modern software paradigms makes it increasingly difficult to derive such models and renders previous performance modeling techniques infeasible. Modern cloud applications are often deployed as a collection of fine-grained and interconnected components called microservices. Microservice architectures offer massive benefits but also have broad implications for the performance characteristics of the respective systems. In addition, the microservices paradigm is typically paired with a DevOps culture, resulting in frequent application and deployment changes. Such applications are often referred to as cloud-native applications. In summary, the increasing use of ever-changing cloud-hosted microservice applications introduces a number of unique challenges for modeling the performance of modern applications. These include the amount, type, and structure of monitoring data, frequent behavioral changes, or infrastructure variabilities. This violates common assumptions of the state of the art and opens a research gap for our work. In this thesis, we present five techniques for automated learning of performance models for cloud-native software systems. We achieve this by combining machine learning with traditional performance modeling techniques. Unlike previous work, our focus is on cloud-hosted and continuously evolving microservice architectures, so-called cloud-native applications. Therefore, our contributions aim to solve the above challenges to deliver automated performance models with minimal computational overhead and no manual intervention. Depending on the cloud computing model, privacy agreements, or monitoring capabilities of each platform, we identify different scenarios where performance modeling, prediction, and optimization techniques can provide great benefits. Specifically, the contributions of this thesis are as follows: Monitorless: Application-agnostic prediction of performance degradations. To manage application performance with only platform-level monitoring, we propose Monitorless, the first truly application-independent approach to detecting performance degradation. We use machine learning to bridge the gap between platform-level monitoring and application-specific measurements, eliminating the need for application-level monitoring. Monitorless creates a single and holistic resource saturation model that can be used for heterogeneous and untrained applications. Results show that Monitorless infers resource-based performance degradation with 97% accuracy. Moreover, it can achieve similar performance to typical autoscaling solutions, despite using less monitoring information. SuanMing: Predicting performance degradation using tracing. We introduce SuanMing to mitigate performance issues before they impact the user experience. This contribution is applied in scenarios where tracing tools enable application-level monitoring. SuanMing predicts explainable causes of expected performance degradations and prevents performance degradations before they occur. Evaluation results show that SuanMing can predict and pinpoint future performance degradations with an accuracy of over 90%. SARDE: Continuous and autonomous estimation of resource demands. We present SARDE to learn application models for highly variable application deployments. This contribution focuses on the continuous estimation of application resource demands, a key parameter of performance models. SARDE represents an autonomous ensemble estimation technique. It dynamically and continuously optimizes, selects, and executes an ensemble of approaches to estimate resource demands in response to changes in the application or its environment. Through continuous online adaptation, SARDE efficiently achieves an average resource demand estimation error of 15.96% in our evaluation. DepIC: Learning parametric dependencies from monitoring data. DepIC utilizes feature selection techniques in combination with an ensemble regression approach to automatically identify and characterize parametric dependencies. Although parametric dependencies can massively improve the accuracy of performance models, DepIC is the first approach to automatically learn such parametric dependencies from passive monitoring data streams. Our evaluation shows that DepIC achieves 91.7% precision in identifying dependencies and reduces the characterization prediction error by 30% compared to the best individual approach. Baloo: Modeling the configuration space of databases. To study the impact of different configurations within distributed DBMSs, we introduce Baloo. Our last contribution models the configuration space of databases considering measurement variabilities in the cloud. More specifically, Baloo dynamically estimates the required benchmarking measurements and automatically builds a configuration space model of a given DBMS. Our evaluation of Baloo on a dataset consisting of 900 configuration points shows that the framework achieves a prediction error of less than 11% while saving up to 80% of the measurement effort. Although the contributions themselves are orthogonally aligned, taken together they provide a holistic approach to performance management of modern cloud-native microservice applications. Our contributions are a significant step forward as they specifically target novel and cloud-native software development and operation paradigms, surpassing the capabilities and limitations of previous approaches. In addition, the research presented in this paper also has a significant impact on the industry, as the contributions were developed in collaboration with research teams from Nokia Bell Labs, Huawei, and Google. Overall, our solutions open up new possibilities for managing and optimizing cloud applications and improve cost and energy efficiency.}, author = {Grohmann, Johannes}, keywords = {t_dissertation descartes statistical_estimation_and_machine_learning prediction performance}, month = {March}, note = {Distinction}, school = {Universität Würzburg}, title = {Model Learning for Performance Prediction of Cloud-native Microservice Applications}, type = {Dissertation}, year = 2022 }

A case study on the stability of performance tests for serverless applications. S. Eismann; D. Costa; L. Liao; C.-P. Bezemer; W. Shang; A. van Hoorn; S. Kounev; in Journal of Systems and Software (JSS) (2022). 189
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.
@article{eismann2022study, abstract = {Context: While in serverless computing, application resource management and operational concerns are generally delegated to the cloud provider, ensuring that serverless applications meet their performance requirements is still a responsibility of the developers. Performance testing is a commonly used performance assessment practice; however, it traditionally requires visibility of the resource environment. Objective: In this study, we investigate whether performance tests of serverless applications are stable, that is, if their results are reproducible, and what implications the serverless paradigm has for performance tests. Method: We conduct a case study where we collect two datasets of performance test results: (a) repetitions of performance tests for varying memory size and load intensities and (b) three repetitions of the same performance test every day for ten months. Results: We find that performance tests of serverless applications are comparatively stable if conducted on the same day. However, we also observe short-term performance variations and frequent long-term performance changes. Conclusion: Performance tests for serverless applications can be stable; however, the serverless model impacts the planning, execution, and analysis of performance tests.}, author = {Eismann, Simon and Costa, Diego and Liao, Lizhi and Bezemer, Cor-Paul and Shang, Weiyi and van Hoorn, André and Kounev, Samuel}, journal = {Journal of Systems and Software (JSS)}, keywords = {descartes t_journalmagazine}, title = {A case study on the stability of performance tests for serverless applications}, volume = 189, year = 2022 }

Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling. M. Straesser; J. Grohmann; J. von Kistowski; S. Eismann; A. Bauer; S. Kounev; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). 105–115.
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.
@inproceedings{10.1145/3489525.3511680, abstract = {Autoscaling is a task of major importance in the cloud computing domain as it directly affects both operating costs and customer experience. Although there has been active research in this area for over ten years now, there is still a significant gap between the proposed methods in the literature and the deployed autoscalers in practice. Hence, many research autoscalers do not find their way into production deployments. This paper describes six core challenges that arise in production systems that are still not solved by most research autoscalers. We illustrate these problems through experiments in a realistic cloud environment with a real-world multi-service business application and show that commonly used autoscalers have various shortcomings. In addition, we analyze the behavior of overloaded services and show that these can be problematic for existing autoscalers. Generally, we analyze that these challenges are only insufficiently addressed in the literature and conclude that future scaling approaches should focus on the needs of production systems.}, address = {New York, NY, USA}, author = {Straesser, Martin and Grohmann, Johannes and von Kistowski, J{ó}akim and Eismann, Simon and Bauer, Andr{é} and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {cloud_computing t_full autoscaling descartes sys:relevantfor:se-group microservices}, pages = {105–115}, publisher = {Association for Computing Machinery}, series = {ICPE '22}, title = {Why Is It Not Solved Yet? Challenges for Production-Ready Autoscaling}, year = 2022 }

SPEC Research Group Newsletter, Issue 11. S. Kounev; A. van Hoorn; M. Straesser; A. Bauer; (2022, April).
Published by Standard Performance Evaluation Corporation (SPEC)
[ BibTeX ] [ Download ]
 
@misc{SPEC-RG-NEWSLETTER-11, author = {Kounev, Samuel and van Hoorn, Andre and Straesser, Martin and Bauer, Andr{é}}, keywords = {t_other descartes sys:relevantfor:se-group SPEC}, month = {April}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {SPEC Research Group Newsletter, Issue 11}, year = 2022 }

A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions. E. Henrichs; V. Lesch; M. Straesser; S. Kounev; C. Krupitzer; in Information and Software Technology (2022). 149 106940.
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.
@article{HENRICHS2022106940, abstract = {Context: Recent developments in modern IT systems including internet of things, edge/fog computing, or cyber–physical systems support intelligent and seamless interaction between users and systems. This requires a reaction to changes in their environment or the system. Adaptive systems provide mechanisms for these reactions. Objective: To implement this functionality, several approaches for the planning of adaptations exist that rely on rules, utility functions, or advanced techniques, such as machine learning. As the adaptation space with possible options is often extensively huge, optimization techniques might support efficient determination of the adaptation space and identify the system’s optimal configuration. With this paper, we provide a systematic review of adaptation planning as the optimization target. Method: In this paper, we review which optimization techniques are applied for adaptation planning in adaptive systems using a systematic literature review approach. Results: We reviewed 115 paper in detail out of an initial search set of 9,588 papers. Our analysis reveals that learning techniques and genetic algorithms are by far dominant; in total, heuristics (anytime learning) are more frequently applied as exact algorithms. We observed that around 57% of the approaches target multi-objectiveness and around 30% integrate distributed optimization. As last dimension, we focused on situation-awareness, which is only supported by two approaches. Conclusion: In this paper, we provide an overview of the current state of the art of approaches that rely on optimization techniques for planning adaptations in adaptive systems and further derive open research challenges, in particular regarding the integration of distributed optimization and situation-awareness.}, author = {Henrichs, Elia and Lesch, Veronika and Straesser, Martin and Kounev, Samuel and Krupitzer, Christian}, journal = {Information and Software Technology}, keywords = {planning adaptation_optimization self_adaptive_systems descartes survey t_journalmagazine sys:relevantfor:se-group}, pages = 106940, title = {A literature review on optimization techniques for adaptation planning in adaptive systems: State of the art and research directions}, volume = 149, year = 2022 }

SPEC Research - Introducing the Predictive Data Analytics Working Group. A. Bauer; M. Leznik; D. Seybold; I. Trubin; B. Erb; J. Domaschka; P. Jamshidi; in Companion of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
[ BibTeX ] [ Download ]
 
@conference{noauthororeditor, author = {Bauer, André and Leznik, Mark and Seybold, Daniel and Trubin, Igor and Erb, Benjamin and Domaschka, Jörg and Jamshidi, Pooyan}, booktitle = {Companion of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {descartes t_poster spec}, month = 4, title = {SPEC Research - Introducing the Predictive Data Analytics Working Group}, year = 2022 }

Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning. A. Bauer; M. Straesser; L. Beierlieb; M. Meissner; S. Kounev; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
[ BibTeX ] [ Download ]
 
@conference{bauer2022automated, author = {Bauer, André and Straesser, Martin and Beierlieb, Lukas and Meissner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {myown timeseries t_short descartes machine_learning}, month = 4, publisher = {ACM}, title = {Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning}, year = 2022 }

FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles. M. Hadry; V. Lesch; S. Kounev; in Companion of the 13th ACM/SPEC International Conference on Performance Engineering (ICPE 2022) (2022).
[ BibTeX ] [ Download ]
 
@inproceedings{hadry2022towards, author = {Hadry, Marius and Lesch, Veronika and Kounev, Samuel}, booktitle = {Companion of the 13th ACM/SPEC International Conference on Performance Engineering (ICPE 2022)}, keywords = {t_visionposition descartes intelligent_transportation_systems cyber-physical_systems}, month = {April}, publisher = {ACM}, title = {FADE: Towards Flexible and Adaptive Distance Estimation Considering Obstacles}, year = 2022 }

An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case. T. Prantl; S. Engel; A. Bauer; A. E. Ben Yahya; S. Herrnleben; L. Iffländer; A. Dmitrienko; S. Kounev; in 2022 IEEE 95th Vehicular Technology Conference (VTC2022 (2022).
[ BibTeX ]
 
@inproceedings{PrEnBaAlHeIfDmKo2022-IoT/CPS-DistributedGroupEncryption, author = {Prantl, Thomas and Engel, Simon and Bauer, Andre and Ben Yahya, Ala Eddine and Herrnleben, Stefan and Iffländer, Lukas and Dmitrienko, Alexandra and Kounev, Samuel}, booktitle = {2022 IEEE 95th Vehicular Technology Conference (VTC2022}, keywords = {descartes t_workshop}, organization = {IEEE}, title = {An Experience Report on the Suitability of a Distributed Group Encryption Scheme for an IoT Use Case}, year = 2022 }

Change Point Detection for MongoDB Time Series Performance Regression. M. Leznik; M. S. Iqbal; I. A. Trubin; A. Lochner; P. Jamshidi; A. Bauer; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
[ BibTeX ]
 
@conference{leznik2022change, author = {Leznik, Mark and Iqbal, Md Shahriar and Trubin, Igor A. and Lochner, Arne and Jamshidi, Pooyan and Bauer, André}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {timeseries t_short descartes machine_learning performance}, month = 4, publisher = {ACM}, title = {Change Point Detection for MongoDB Time Series Performance Regression}, year = 2022 }

SPEC Efficiency Benchmark Development: How to Contribute to the Future of Energy Conservation. M. Meissner; K.-D. Lange; J. Arnold; S. Sharma; R. Tipley; N. Rawtani; D. Reiner; M. Petrich; A. Cragin; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022).
[ Abstract ] [ BibTeX ] [ Download ]
 
A driving force behind the improvement of server efficiency in recent years is the use of SPEC benchmarks. They are used in mandatory government regulations, the ISO/IEC 21836:2020 standard, and product marketing, giving server manufacturers and buyers significant incentive to improve energy efficiency. To produce relevant results, benchmarks need to take into account future trends in hardware and software development, such as the introduction of new accelerators and workloads. To keep pace with the development of the fast moving IT landscape, SPEC plans to introduce a workload bounty program to encourage researchers to develop novel workloads. Submitted workloads will be considered for inclusion in future SPEC Efficiency benchmarks and rewarded. In this paper, we outline the process of energy-efficiency benchmark development. SPEC ensures the development of high-quality benchmarks for government regulations through its extensive experience and collaboration with stakeholders from industry, academia, and governments. One of the tools that emerged from this process is the Chauffeur Worklet Development Kit (WDK), which can be used by researchers to develop next-generation workloads to enhance the real-world relevance of future SPEC benchmarks, a critical element for the benchmarks to contribute to future energy conservation.
@inproceedings{meissner2022efficiency, abstract = {A driving force behind the improvement of server efficiency in recent years is the use of SPEC benchmarks. They are used in mandatory government regulations, the ISO/IEC 21836:2020 standard, and product marketing, giving server manufacturers and buyers significant incentive to improve energy efficiency. To produce relevant results, benchmarks need to take into account future trends in hardware and software development, such as the introduction of new accelerators and workloads. To keep pace with the development of the fast moving IT landscape, SPEC plans to introduce a workload bounty program to encourage researchers to develop novel workloads. Submitted workloads will be considered for inclusion in future SPEC Efficiency benchmarks and rewarded. In this paper, we outline the process of energy-efficiency benchmark development. SPEC ensures the development of high-quality benchmarks for government regulations through its extensive experience and collaboration with stakeholders from industry, academia, and governments. One of the tools that emerged from this process is the Chauffeur Worklet Development Kit (WDK), which can be used by researchers to develop next-generation workloads to enhance the real-world relevance of future SPEC benchmarks, a critical element for the benchmarks to contribute to future energy conservation.}, address = {New York, NY, USA}, author = {Meissner, Maximilian and Lange, Klaus-Dieter and Arnold, Jeremy and Sharma, Sanjay and Tipley, Roger and Rawtani, Nishant and Reiner, David and Petrich, Mike and Cragin, Aaron}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {efficiency descartes t_tutorial benchmark spec}, month = {April}, publisher = {ACM}, title = {SPEC Efficiency Benchmark Development: How to Contribute to the Future of Energy Conservation}, year = 2022 }

2021 [ nach oben ]

SPEC Research Group Newsletter, vol. 3 no. 2. S. Kounev; A. van Hoorn; A. Bauer; (2021, April).
Published by Standard Performance Evaluation Corporation (SPEC)
[ BibTeX ] [ URL ] [ Download ]
 
@misc{SPEC-RG-NEWSLETTER-10, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{é}}, keywords = {t_other descartes SPEC}, month = {April}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 3 no. 2}}, year = 2021 }

A Case Study on Optimization of Warehouses V. Lesch; P. B. Müller; M. Krämer; S. Kounev; C. Krupitzer; (2021).
[ BibTeX ] [ URL ] [ Download ]
 
@techreport{lesch2021study, author = {Lesch, Veronika and Müller, Patrick B.M. and Krämer, Moritz and Kounev, Samuel and Krupitzer, Christian}, keywords = {descartes cyber-physical_systems logistics t_techreport}, title = {A Case Study on Optimization of Warehouses}, year = 2021 }

Container Start Times: Empirical Analysis and Predictability M. Straesser; (2021).
[ BibTeX ] [ URL ] [ Download ]
 
@presentation{straesser2021container, author = {Straesser, Martin}, keywords = {container t_talk descartes empirical_study}, title = {Container Start Times: Empirical Analysis and Predictability}, year = 2021 }

Attack-aware Security Function Management. L. Iffländer; (2021, January).
[ Abstract ] [ BibTeX ] [ Download ]
 
Over the last decades, cybersecurity has become an increasingly important issue. Between 2019 and 2011 alone, the losses from cyberattacks in the United States grew by 6217%. At the same time, attacks became not only more intensive but also more and more versatile and diverse. Cybersecurity has become everyone’s concern. Today, service providers require sophisticated and extensive security infrastructures comprising many security functions dedicated to various cyberattacks. Still, attacks become more violent to a level where infrastructures can no longer keep up. Simply scaling up is no longer sufficient. To address this challenge, in a whitepaper, the Cloud Security Alliance (CSA) proposed multiple work packages for security infrastructure, leveraging the possibilities of Software-defined Networking (SDN) and Network Function Virtualization (NFV). Security functions require a more sophisticated modeling approach than regular network functions. Notably, the property to drop packets deemed malicious has a significant impact on Security Service Function Chains (SSFCs)—service chains consisting of multiple security functions to protect against multiple at- tack vectors. Under attack, the order of these chains influences the end-to-end system performance depending on the attack type. Unfortunately, it is hard to predict the attack composition at system design time. Thus, we make a case for dynamic attack-aware SSFC reordering. Also, we tackle the issues of the lack of integration between security functions and the surrounding network infrastructure, the insufficient use of short term CPU frequency boosting, and the lack of Intrusion Detection and Prevention Systems (IDPS) against database ransomware attacks. Current works focus on characterizing the performance of security functions and their behavior under overload without considering the surrounding infrastructure. Other works aim at replacing security functions using network infrastructure features but do not consider integrating security functions within the network. Further publications deal with using SDN for security or how to deal with new vulnerabilities introduced through SDN. However, they do not take security function performance into account. NFV is a popular field for research dealing with frameworks, benchmarking methods, the combination with SDN, and implementing security functions as Virtualized Network Functions (VNFs). Research in this area brought forth the concept of Service Function Chains (SFCs) that chain multiple network functions after one another. Nevertheless, they still do not consider the specifics of security functions. The mentioned CSA whitepaper proposes many valuable ideas but leaves their realization open to others. This thesis presents solutions to increase the performance of single security functions using SDN, performance modeling, a framework for attack-aware SSFC reordering, a solution to make better use of CPU frequency boosting, and an IDPS against database ransomware. Specifically, the primary contributions of this work are: • We present approaches to dynamically bypass Intrusion Detection Systems (IDS) in order to increase their performance without reducing the security level. To this end, we develop and implement three SDN-based approaches (two dynamic and one static). We evaluate the proposed approaches regarding security and performance and show that they significantly increase the performance com- pared to an inline IDS without significant security deficits. We show that using software switches can further increase the performance of the dynamic approaches up to a point where they can eliminate any throughput drawbacks when using the IDS. • We design a DDoS Protection System (DPS) against TCP SYN flood at tacks in the form of a VNF that works inside an SDN-enabled network. This solution eliminates known scalability and performance drawbacks of existing solutions for this attack type. Then, we evaluate this solution showing that it correctly handles the connection establishment and present solutions for an observed issue. Next, we evaluate the performance showing that our solution increases performance up to three times. Parallelization and parameter tuning yields another 76% performance boost. Based on these findings, we discuss optimal deployment strategies. • We introduce the idea of attack-aware SSFC reordering and explain its impact in a theoretical scenario. Then, we discuss the required information to perform this process. We validate our claim of the importance of the SSFC order by analyzing the behavior of single security functions and SSFCs. Based on the results, we conclude that there is a massive impact on the performance up to three orders of magnitude, and we find contradicting optimal orders for different workloads. Thus, we demonstrate the need for dynamic reordering. Last, we develop a model for SSFC regarding traffic composition and resource demands. We classify the traffic into multiple classes and model the effect of single security functions on the traffic and their generated resource demands as functions of the incoming network traffic. Based on our model, we propose three approaches to determine optimal orders for reordering. • We implement a framework for attack-aware SSFC reordering based on this knowledge. The framework places all security functions inside an SDN-enabled network and reorders them using SDN flows. Our evaluation shows that the framework can enforce all routes as desired. It correctly adapts to all attacks and returns to the original state after the attacks cease. We find possible security issues at the moment of reordering and present solutions to eliminate them. • Next, we design and implement an approach to load balance servers while taking into account their ability to go into a state of Central Processing Unit (CPU) frequency boost. To this end, the approach collects temperature information from available hosts and places services on the host that can attain the boosted mode the longest. We evaluate this approach and show its effectiveness. For high load scenarios, the approach increases the overall performance and the performance per watt. Even better results show up for low load workloads, where not only all performance metrics improve but also the temperatures and total power consumption decrease. • Last, we design an IDPS protecting against database ransomware attacks that comprise multiple queries to attain their goal. Our solution models these attacks using a Colored Petri Net (CPN). A proof-of-concept implementation shows that our approach is capable of detecting attacks without creating false positives for benign scenarios. Furthermore, our solution creates only a small performance impact. Our contributions can help to improve the performance of security infrastructures. We see multiple application areas from data center operators over software and hardware developers to security and performance researchers. Most of the above-listed contributions found use in several research publications. Regarding future work, we see the need to better integrate SDN-enabled security functions and SSFC reordering in data center networks. Future SSFC should discriminate between different traffic types, and security frameworks should support automatically learning models for security functions. We see the need to consider energy efficiency when regarding SSFCs and take CPU boosting technologies into account when designing performance models as well as placement, scaling, and deployment strategies. Last, for a faster adaptation against recent ransomware attacks, we propose machine-assisted learning for database IDPS signatures.
@phdthesis{ifflander2021attackaware, abstract = {Over the last decades, cybersecurity has become an increasingly important issue. Between 2019 and 2011 alone, the losses from cyberattacks in the United States grew by 6217%. At the same time, attacks became not only more intensive but also more and more versatile and diverse. Cybersecurity has become everyone’s concern. Today, service providers require sophisticated and extensive security infrastructures comprising many security functions dedicated to various cyberattacks. Still, attacks become more violent to a level where infrastructures can no longer keep up. Simply scaling up is no longer sufficient. To address this challenge, in a whitepaper, the Cloud Security Alliance (CSA) proposed multiple work packages for security infrastructure, leveraging the possibilities of Software-defined Networking (SDN) and Network Function Virtualization (NFV). Security functions require a more sophisticated modeling approach than regular network functions. Notably, the property to drop packets deemed malicious has a significant impact on Security Service Function Chains (SSFCs)—service chains consisting of multiple security functions to protect against multiple at- tack vectors. Under attack, the order of these chains influences the end-to-end system performance depending on the attack type. Unfortunately, it is hard to predict the attack composition at system design time. Thus, we make a case for dynamic attack-aware SSFC reordering. Also, we tackle the issues of the lack of integration between security functions and the surrounding network infrastructure, the insufficient use of short term CPU frequency boosting, and the lack of Intrusion Detection and Prevention Systems (IDPS) against database ransomware attacks. Current works focus on characterizing the performance of security functions and their behavior under overload without considering the surrounding infrastructure. Other works aim at replacing security functions using network infrastructure features but do not consider integrating security functions within the network. Further publications deal with using SDN for security or how to deal with new vulnerabilities introduced through SDN. However, they do not take security function performance into account. NFV is a popular field for research dealing with frameworks, benchmarking methods, the combination with SDN, and implementing security functions as Virtualized Network Functions (VNFs). Research in this area brought forth the concept of Service Function Chains (SFCs) that chain multiple network functions after one another. Nevertheless, they still do not consider the specifics of security functions. The mentioned CSA whitepaper proposes many valuable ideas but leaves their realization open to others. This thesis presents solutions to increase the performance of single security functions using SDN, performance modeling, a framework for attack-aware SSFC reordering, a solution to make better use of CPU frequency boosting, and an IDPS against database ransomware. Specifically, the primary contributions of this work are: • We present approaches to dynamically bypass Intrusion Detection Systems (IDS) in order to increase their performance without reducing the security level. To this end, we develop and implement three SDN-based approaches (two dynamic and one static). We evaluate the proposed approaches regarding security and performance and show that they significantly increase the performance com- pared to an inline IDS without significant security deficits. We show that using software switches can further increase the performance of the dynamic approaches up to a point where they can eliminate any throughput drawbacks when using the IDS. • We design a DDoS Protection System (DPS) against TCP SYN flood at tacks in the form of a VNF that works inside an SDN-enabled network. This solution eliminates known scalability and performance drawbacks of existing solutions for this attack type. Then, we evaluate this solution showing that it correctly handles the connection establishment and present solutions for an observed issue. Next, we evaluate the performance showing that our solution increases performance up to three times. Parallelization and parameter tuning yields another 76% performance boost. Based on these findings, we discuss optimal deployment strategies. • We introduce the idea of attack-aware SSFC reordering and explain its impact in a theoretical scenario. Then, we discuss the required information to perform this process. We validate our claim of the importance of the SSFC order by analyzing the behavior of single security functions and SSFCs. Based on the results, we conclude that there is a massive impact on the performance up to three orders of magnitude, and we find contradicting optimal orders for different workloads. Thus, we demonstrate the need for dynamic reordering. Last, we develop a model for SSFC regarding traffic composition and resource demands. We classify the traffic into multiple classes and model the effect of single security functions on the traffic and their generated resource demands as functions of the incoming network traffic. Based on our model, we propose three approaches to determine optimal orders for reordering. • We implement a framework for attack-aware SSFC reordering based on this knowledge. The framework places all security functions inside an SDN-enabled network and reorders them using SDN flows. Our evaluation shows that the framework can enforce all routes as desired. It correctly adapts to all attacks and returns to the original state after the attacks cease. We find possible security issues at the moment of reordering and present solutions to eliminate them. • Next, we design and implement an approach to load balance servers while taking into account their ability to go into a state of Central Processing Unit (CPU) frequency boost. To this end, the approach collects temperature information from available hosts and places services on the host that can attain the boosted mode the longest. We evaluate this approach and show its effectiveness. For high load scenarios, the approach increases the overall performance and the performance per watt. Even better results show up for low load workloads, where not only all performance metrics improve but also the temperatures and total power consumption decrease. • Last, we design an IDPS protecting against database ransomware attacks that comprise multiple queries to attain their goal. Our solution models these attacks using a Colored Petri Net (CPN). A proof-of-concept implementation shows that our approach is capable of detecting attacks without creating false positives for benign scenarios. Furthermore, our solution creates only a small performance impact. Our contributions can help to improve the performance of security infrastructures. We see multiple application areas from data center operators over software and hardware developers to security and performance researchers. Most of the above-listed contributions found use in several research publications. Regarding future work, we see the need to better integrate SDN-enabled security functions and SSFC reordering in data center networks. Future SSFC should discriminate between different traffic types, and security frameworks should support automatically learning models for security functions. We see the need to consider energy efficiency when regarding SSFCs and take CPU boosting technologies into account when designing performance models as well as placement, scaling, and deployment strategies. Last, for a faster adaptation against recent ransomware attacks, we propose machine-assisted learning for database IDPS signatures.}, author = {Iffl{ä}nder, Lukas}, keywords = {t_dissertation descartes performance sys:relevantfor:se-group security}, month = {01}, school = {Universit{ä}t W{ü}rzburg}, title = {Attack-aware Security Function Management}, type = {doctoralthesis}, year = 2021 }

Towards a Mobile Robot Localization Benchmark with Challenging Sensordata in an Industrial Environment. F. Spieß; J. Friesslich; D. Bluemm; F. Mast; D. Vinokour; S. Kounev; T. Kaupp; N. Strobel; in 2021 20th International Conference on Advanced Robotics (ICAR) (2021). 857–864.
[ Abstract ] [ BibTeX ] [ URL ]
 
To arrive at a realistic assessment of localization methods in terms of their performance in an industrial environment under various challenging conditions, we provide a benchmark to evaluate algorithms both for individual components as well as multi-sensor systems. For several sensor types, including wheel odometry, RGB cameras, RGB-D cameras, and LIDAR, potential issues were identified. The accuracy of wheel odometry, for example, when there are bumps on the track. For each sensor type, we explicitly chose a track for the benchmark dataset containing situations where the sensor fails to provide adequate measurements. Based on the acquired sensor data, localization can be achieved either using a single sensor information or sensor fusion. To help evaluate the output of associated localization algorithms, we provide a software to evaluate a set of metrics as part of the paper. An example application of the benchmark with state-of-the-art algorithms for each sensor is also provided.
@inproceedings{9659355, abstract = {To arrive at a realistic assessment of localization methods in terms of their performance in an industrial environment under various challenging conditions, we provide a benchmark to evaluate algorithms both for individual components as well as multi-sensor systems. For several sensor types, including wheel odometry, RGB cameras, RGB-D cameras, and LIDAR, potential issues were identified. The accuracy of wheel odometry, for example, when there are bumps on the track. For each sensor type, we explicitly chose a track for the benchmark dataset containing situations where the sensor fails to provide adequate measurements. Based on the acquired sensor data, localization can be achieved either using a single sensor information or sensor fusion. To help evaluate the output of associated localization algorithms, we provide a software to evaluate a set of metrics as part of the paper. An example application of the benchmark with state-of-the-art algorithms for each sensor is also provided.}, author = {Spieß, Florian and Friesslich, Jonas and Bluemm, Daniel and Mast, Fabio and Vinokour, Dmitrij and Kounev, Samuel and Kaupp, Tobias and Strobel, Norbert}, booktitle = {2021 20th International Conference on Advanced Robotics (ICAR)}, keywords = {descartes sys:relevantfor:se-group}, month = {dec}, pages = {857-864}, title = {Towards a Mobile Robot Localization Benchmark with Challenging Sensordata in an Industrial Environment}, year = 2021 }

The State of Serverless Applications: Collection, Characterization, and Community Consensus. S. Eismann; J. Scheuner; E. van Eyk; M. Schwinger; J. Grohmann; N. Herbst; C. Abad; A. Iosup; in Transactions on Software Engineering (2021).
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
Over the last five years, all major cloud platform providers have increased their serverless offerings. Many early adopters report significant benefits for serverless-based over traditional applications, and many companies are considering moving to serverless themselves. However, currently there exist only few, scattered, and sometimes even conflicting reports on when serverless applications are well suited and what the best practices for their implementation are. We address this problem in the present study about the state of serverless applications. We collect descriptions of 89 serverless applications from open-source projects, academic literature, industrial literature, and domain-specific feedback. We analyze 16 characteristics that describe why and when successful adopters are using serverless applications, and how they are building them. We further compare the results of our characterization study to 10 existing, mostly industrial, studies and datasets; this allows us to identify points of consensus across multiple studies, investigate points of disagreement, and overall confirm the validity of our results. The results of this study can help managers to decide if they should adopt serverless technology, engineers to learn about current practices of building serverless applications, and researchers and platform providers to better understand the current landscape of serverless applications.
@article{eismann2021state, abstract = {Over the last five years, all major cloud platform providers have increased their serverless offerings. Many early adopters report significant benefits for serverless-based over traditional applications, and many companies are considering moving to serverless themselves. However, currently there exist only few, scattered, and sometimes even conflicting reports on when serverless applications are well suited and what the best practices for their implementation are. We address this problem in the present study about the state of serverless applications. We collect descriptions of 89 serverless applications from open-source projects, academic literature, industrial literature, and domain-specific feedback. We analyze 16 characteristics that describe why and when successful adopters are using serverless applications, and how they are building them. We further compare the results of our characterization study to 10 existing, mostly industrial, studies and datasets; this allows us to identify points of consensus across multiple studies, investigate points of disagreement, and overall confirm the validity of our results. The results of this study can help managers to decide if they should adopt serverless technology, engineers to learn about current practices of building serverless applications, and researchers and platform providers to better understand the current landscape of serverless applications.}, author = {Eismann, Simon and Scheuner, Joel and van Eyk, Erwin and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina and Iosup, Alexandru}, journal = {Transactions on Software Engineering}, keywords = {descartes t_journalmagazine}, title = {The State of Serverless Applications: Collection, Characterization, and Community Consensus}, year = 2021 }

Security and Privacy Aspects of Digital Contact Tracing. F. Roos; Thesis; University of Würzburg. (2021, October).
[ BibTeX ]
 
@mastersthesis{roos2021security, author = {Roos, Filipp}, keywords = {tracing thesis_supervised_by_sss_member privacy thesis_supervised_by_SSS_member contact sss-group security}, month = {oct}, school = {University of W{ü}rzburg}, title = {Security and Privacy Aspects of Digital Contact Tracing}, type = {Master Thesis}, year = 2021 }

Towards Splitting Monolithic Workflows into Serverless Functions and Estimating Their Run-Time in the Earth Observation Domain. D. Kaiser; B. D. Simon; A. Bauer; S. Kounev; in Proceedings of the 12th Symposium on Software Performance (SSP) (2021).
Extended Abstract
[ BibTeX ]
 
@inproceedings{kaiser2021split, author = {Kaiser, Dennis and Simon, Bohdan Dovhan and Bauer, Andr{é} and Kounev, Samuel}, booktitle = {Proceedings of the 12th Symposium on Software Performance (SSP)}, keywords = {t_extendedabstract descartes severless}, month = {November}, note = {Extended Abstract}, title = {{Towards Splitting Monolithic Workflows into Serverless Functions and Estimating Their Run-Time in the Earth Observation Domain}}, year = 2021 }

ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations. S. Herrnleben; M. Leidinger; V. Lesch; T. Prantl; J. Grohmann; C. Krupitzer; S. Kounev; in Performance Evaluation Methodologies and Tools, Q. Zhao, L. Xia (eds.) (2021). 72–92.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.
@inproceedings{10.1007/978-3-030-92511-6_5, abstract = {Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Leidinger, Maximilian and Lesch, Veronika and Prantl, Thomas and Grohmann, Johannes and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Performance Evaluation Methodologies and Tools}, editor = {Zhao, Qianchuan and Xia, Li}, keywords = {t_full descartes networking}, pages = {72–92}, publisher = {Springer International Publishing}, title = {ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations}, year = 2021 }

Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms. V. Lesch; M. König; S. Kounev; A. Stein; C. Krupitzer; in Applied Intelligence (2021).
[ BibTeX ] [ DOI ]
 
@article{lesch2021tackling, author = {Lesch, Veronika and König, Maximilian and Kounev, Samuel and Stein, Anthony and Krupitzer, Christian}, journal = {Applied Intelligence}, keywords = {optimization descartes intelligent_transportation_systems cyber-physical_systems t_journalmagazine}, title = {Tackling the Rich Vehicle Routing Problem with Nature-Inspired Algorithms}, year = 2021 }

A Case Study on Optimization of Platooning Coordination V. Lesch; M. Hadry; S. Kounev; C. Krupitzer; (2021).
[ BibTeX ] [ URL ] [ Download ]
 
@techreport{lesch2021studyplatooning, author = {Lesch, Veronika and Hadry, Marius and Kounev, Samuel and Krupitzer, Christian}, institution = {Universit{ä}t W{ü}rzburg and Universit{ä}t Hohenheim}, keywords = {descartes intelligent_transportation_systems cyber-physical_systems t_techreport}, title = {A Case Study on Optimization of Platooning Coordination}, year = 2021 }

All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers. C. Hagen; C. Weinert; C. Sendner; A. Dmitrienko; T. Schneider; in Network and Distributed System Security Symposium (NDSS) (2021).
(To Appear)
[ BibTeX ] [ URL ]
 
@inproceedings{DBLP:conf/ndss/2021, author = {Hagen, Christoph and Weinert, Christian and Sendner, Christoph and Dmitrienko, Alexandra and Schneider, Thomas}, booktitle = {Network and Distributed System Security Symposium (NDSS)}, keywords = {accepted sss-group dblp}, note = {(To Appear)}, publisher = {The Internet Society}, title = {All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers}, year = 2021 }

A Case Study of Vehicle Route Optimization V. Lesch; M. König; S. Kounev; A. Stein; C. Krupitzer; (2021).
[ BibTeX ] [ URL ] [ Download ]
 
@techreport{lesch2021studyvrp, author = {Lesch, Veronika and König, Maximilian and Kounev, Samuel and Stein, Anthony and Krupitzer, Christian}, institution = {Universit{ä}t W{ü}rzburg and PASS Logistics Solutions AG and Universit{ä}t Hohenheim}, keywords = {descartes intelligent_transportation_systems cyber-physical_systems t_techreport}, title = {A Case Study of Vehicle Route Optimization}, year = 2021 }

Vision: Challenges & Opportunities. M. Annaaram; N. Asokan; B. G. Atli; S. Avestimeh; F. Brasser; R. Cammarota; A. Dmitrienko; A. Dziedzic; T. Given-Wilson; L. J. Gunn; F. Kerschbaum; F. Koushanfar; A. Legay; M. Miettinen; T. D. Nguyen; N. Papernot; A.-R. Sadeghi; M. Schunter; R. Shokri; V. Smith; in Vision Paper of the Private AI Collaborative Research Institute (2021).
[ BibTeX ]
 
@inproceedings{annaaramvision, author = {Annaaram, Murali and Asokan, N. and Atli, Buse Gül and Avestimeh, Salman and Brasser, Ferdinand and Cammarota, Rosario and Dmitrienko, Alexandra and Dziedzic, Adam and Given-Wilson, Thomas and Gunn, Lachlan J. and Kerschbaum, Florian and Koushanfar, Farinaz and Legay, Axel and Miettinen, Markus and Nguyen, Thien Duc and Papernot, Nicolas and Sadeghi, Ahmad-Reza and Schunter, Matthias and Shokri, Reza and Smith, Virginia}, booktitle = {Vision Paper of the Private AI Collaborative Research Institute}, keywords = {machine_learning privacy international-conference-workshop-papers-book-chapters sss-group}, title = {Vision: Challenges & Opportunities.}, year = 2021 }

People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot. F. Spiess; L. Reinhart; N. Strobel; D. Kaiser; S. Kounev; T. Kaupp; in Journal of Image and Graphics, (J. of Image; Graphics, eds.) (2021). 9(4) 135–139.
[ BibTeX ]
 
@article{spiess2021people, author = {Spiess, Florian and Reinhart, Lucas and Strobel, Norbert and Kaiser, Dennis and Kounev, Samuel and Kaupp, Tobias}, editor = {of Image, Journal and Graphics}, journal = {Journal of Image and Graphics}, keywords = {descartes t_interdisciplinary t_journalmagazine sys:relevantfor:se-group}, month = {December 2021}, number = 4, pages = {135-139}, title = {People Detection with Depth Silhouettes and Convolutional Neural Networks on a Mobile Robot}, volume = 9, year = 2021 }

Increasing Security in Satellite Networks. K. Schilling; A. Dmitrienko; in 72nd International Astronautical Congress (IAC) (2021).
[ BibTeX ]
 
@inproceedings{schilling2021increasing, author = {Schilling, Klaus and Dmitrienko, Alexandra}, booktitle = {72nd International Astronautical Congress (IAC)}, keywords = {sss-group satellite network security}, title = {Increasing Security in Satellite Networks}, year = 2021 }

Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction. M. Züfle; F. Erhard; S. Kounev; in 20th IEEE International Conference on Machine Learning and Applications (ICMLA) (2021).
(to appear)
[ BibTeX ]
 
@inproceedings{zufle2021machine, author = {Z{ü}fle, Marwin and Erhard, Florian and Kounev, Samuel}, booktitle = {20th IEEE International Conference on Machine Learning and Applications (ICMLA)}, keywords = {t_full descartes statistical_estimation_and_machine_learning prediction online_monitoring_and_forecasting}, note = {(to appear)}, title = {Machine Learning Model Update Strategies for Hard Disk Drive Failure Prediction}, year = 2021 }

An Overview on Approaches for Coordination of Platoons. V. Lesch; M. Breitbach; M. Segata; C. Becker; S. Kounev; C. Krupitzer; in IEEE Transactions on Intelligent Transportation Systems (2021).
Early Access on IEEE Xplore
[ BibTeX ] [ Download ]
 
@article{lesch2021overview, author = {Lesch, Veronika and Breitbach, Martin and Segata, Michele and Becker, Christian and Kounev, Samuel and Krupitzer, Christian}, journal = {IEEE Transactions on Intelligent Transportation Systems}, keywords = {descartes intelligent_transportation_systems cyber-physical_systems internet_of_things t_journalmagazine platooning}, note = {Early Access on IEEE Xplore}, title = {An Overview on Approaches for Coordination of Platoons}, year = 2021 }

Serverless Computing (Dagstuhl Seminar 21201). C. Abad; I. T. Foster; N. Herbst; A. Iosup; in Dagstuhl Reports, (C. Abad; I. T. Foster; N. Herbst; A. Iosup, eds.) (2021). 11(4) 34–93.
[ BibTeX ] [ URL ]
 
@article{abad_et_al:DagRep.11.4.34, address = {Dagstuhl, Germany}, annote = {Keywords: Cloud computing, Cloud continuum, data-driven, design patterns, DevOps, experimentation, model-driven, serverless computing, simulation, software architecture, systems management, vision}, author = {Abad, Cristina and Foster, Ian T. and Herbst, Nikolas and Iosup, Alexandru}, editor = {Abad, Cristina and Foster, Ian T. and Herbst, Nikolas and Iosup, Alexandru}, journal = {Dagstuhl Reports}, keywords = {cloud descartes metrics_and_benchmarking_methodologies elasticity sys:relevantfor:se-group resource_management}, number = 4, pages = {34–93}, publisher = {Schloss Dagstuhl – Leibniz-Zentrum f{ü}r Informatik}, title = {{Serverless Computing (Dagstuhl Seminar 21201)}}, volume = 11, year = 2021 }

Remote Attestation for IoT with Smart Verifier. L. Petzi; Thesis; University of Würzburg. (2021, January).
[ Abstract ] [ BibTeX ]
 
The Internet is evolving. The Internet of Things (IoT) is a disruptive technology, billions of small and smart devices are encompassing every aspect of our lives, which leads us to a new era of the Internet. IoT connects billions of heterogeneous embedded devices in large networks. Restricted capabilities of the devices in combination with various mutually mistrusting parties throughout the network pose new challenges to security. As IoT devices become increasingly important, so does trust in their secure and reliable operation. But before we can trust them, we need to establish the trust. In the past, remote attestation has emerged into a very powerful tool in order to establish trust between devices. Traditional remote attestation establishes a static trust between two devices. This approach suffers badly heterogeneity of IoT network and purely scales to a large number of devices. Furthermore, it is very vulnerable to Denial of Service attacks. Within this thesis, we present a new approach to Remote Attestation in IoT networks. Our design combines the core strength of Remote Attestation with the advantages blockchain technology in order to enable attestation and validation of attestation evidences in a scalable manner. At rst, we develop a remote attestation scheme on a low end device serving as an exemplary IoT device. Second, we design our smart verifier, a flexible system capable of validating and storing attestation evidence. The system is build on top of blockchain technology and solves several issues accompanied by Traditional Remote Attestation in IoT such as scalability and device heterogeneity. In the end, we build and evaluate a proof of concept implementation based on ARM TrustZone and Sawtooth Hyperledger. The proposed system architecture enables trust establishment in a scalable manner even in heterogeneous networks while preventing denial of service attacks
@mastersthesis{chan2019remote, abstract = {The Internet is evolving. The Internet of Things (IoT) is a disruptive technology, billions of small and smart devices are encompassing every aspect of our lives, which leads us to a new era of the Internet. IoT connects billions of heterogeneous embedded devices in large networks. Restricted capabilities of the devices in combination with various mutually mistrusting parties throughout the network pose new challenges to security. As IoT devices become increasingly important, so does trust in their secure and reliable operation. But before we can trust them, we need to establish the trust. In the past, remote attestation has emerged into a very powerful tool in order to establish trust between devices. Traditional remote attestation establishes a static trust between two devices. This approach suffers badly heterogeneity of IoT network and purely scales to a large number of devices. Furthermore, it is very vulnerable to Denial of Service attacks. Within this thesis, we present a new approach to Remote Attestation in IoT networks. Our design combines the core strength of Remote Attestation with the advantages blockchain technology in order to enable attestation and validation of attestation evidences in a scalable manner. At rst, we develop a remote attestation scheme on a low end device serving as an exemplary IoT device. Second, we design our smart verifier, a flexible system capable of validating and storing attestation evidence. The system is build on top of blockchain technology and solves several issues accompanied by Traditional Remote Attestation in IoT such as scalability and device heterogeneity. In the end, we build and evaluate a proof of concept implementation based on ARM TrustZone and Sawtooth Hyperledger. The proposed system architecture enables trust establishment in a scalable manner even in heterogeneous networks while preventing denial of service attacks}, author = {Petzi, Lukas}, booktitle = {CVPR Workshops}, keywords = {remote-attestation thesis_supervised_by_sss_member thesis_supervised_by_SSS_member sss-group IoT security}, month = {jan}, school = {University of W{ü}rzburg}, title = {Remote Attestation for IoT with Smart Verifier}, type = {Bachelor Thesis}, year = 2021 }

The integration of training and off-training activities substantially alters training volume and load analysis in elite rowers. G. Treff; R. Leppich; K. Winkert; J. M. Steinacker; B. Mayer; B. Sperlich; in Scientific reports (2021). 11(1) 1–10.
[ BibTeX ]
 
@article{treff2021integration, author = {Treff, Gunnar and Leppich, Robert and Winkert, Kay and Steinacker, J{ü}rgen M and Mayer, Benjamin and Sperlich, Billy}, journal = {Scientific reports}, keywords = {descartes t_interdisciplinary t_journalmagazine}, number = 1, pages = {1–10}, publisher = {Nature Publishing Group}, title = {The integration of training and off-training activities substantially alters training volume and load analysis in elite rowers}, volume = 11, year = 2021 }

Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT. T. Prantl; D. Prantl; A. Bauer; L. Iffländer; A. Dmitrenko; C. Krupitzer; S. Kounev; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
[ BibTeX ]
 
@inproceedings{PrPrBaIfDmKrKo2021-IPCCC-QuantumGroupEncryption, author = {Prantl, Thomas and Prantl, Dominik and Bauer, André and Iffländer, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {t_full descartes sss-group}, title = {Benchmarking of Pre- and Post-Quantum Group Encryption Schemes with Focus on IoT}, year = 2021 }

Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies. V. Lesch; T. Noack; J. Hefter; S. Kounev; C. Krupitzer; in Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021 (2021).
Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%
[ BibTeX ] [ Download ]
 
@inproceedings{lesch2021towards, author = {Lesch, Veronika and Noack, Tanja and Hefter, Johannes and Kounev, Samuel and Krupitzer, Christian}, booktitle = {Proceedings of the 2nd IEEE International Conference on Autonomic Computing and Self-Organizing Systems - ACSOS 2021}, keywords = {t_full descartes cyber-physical_systems internet_of_things}, note = {Best Paper Award Runner Up, Full Paper Acceptance Ratio: 25%}, publisher = {IEEE}, title = {Towards Situation-Aware Meta-Optimization of Adaptation Planning Strategies}, year = 2021 }

ESCORT: Ethereum Smart COntRacTs Vulnerability Detection using Deep Neural Network and Transfer Learning. O. Lutz; H. Chen; H. Fereidooni; C. Sendner; A. Dmitrienko; A. R. Sadeghi; F. Koushanfar; in ArXiv | arXiv:2103.12607v1 (2021).
[ Abstract ] [ BibTeX ] [ URL ]
 
Ethereum smart contracts are automated decentralized applications on the blockchain that describe the terms of the agreement between buyers and sellers, reducing the need for trusted intermediaries and arbitration. However, the deployment of smart contracts introduces new attack vectors into the cryptocurrency systems. In particular, programming flaws in smart contracts can be and have already been exploited to gain enormous financial profits. It is thus an emerging yet crucial issue to detect vulnerabilities of different classes in contracts in an efficient manner. Existing machine learning-based vulnerability detection methods are limited and only inspect whether the smart contract is vulnerable, or train individual classifiers for each specific vulnerability, or demonstrate multi-class vulnerability detection without extensibility consideration. To overcome the scalability and generalization limitations of existing works, we propose ESCORT, the first Deep Neural Network (DNN)-based vulnerability detection framework for Ethereum smart contracts that support lightweight transfer learning on unseen security vulnerabilities, thus is extensible and generalizable. ESCORT leverages a multi-output NN architecture that consists of two parts: (i) A common feature extractor that learns the semantics of the input contract; (ii) Multiple branch structures where each branch learns a specific vulnerability type based on features obtained from the feature extractor. Experimental results show that ESCORT achieves an average F1-score of 95% on six vulnerability types and the detection time is 0.02 seconds per contract. When extended to new vulnerability types, ESCORT yields an average F1-score of 93%. To the best of our knowledge, ESCORT is the first framework that enables transfer learning on new vulnerability types with minimal modification of the DNN model architecture and re-training overhead.
@article{lutz2021escort, abstract = {Ethereum smart contracts are automated decentralized applications on the blockchain that describe the terms of the agreement between buyers and sellers, reducing the need for trusted intermediaries and arbitration. However, the deployment of smart contracts introduces new attack vectors into the cryptocurrency systems. In particular, programming flaws in smart contracts can be and have already been exploited to gain enormous financial profits. It is thus an emerging yet crucial issue to detect vulnerabilities of different classes in contracts in an efficient manner. Existing machine learning-based vulnerability detection methods are limited and only inspect whether the smart contract is vulnerable, or train individual classifiers for each specific vulnerability, or demonstrate multi-class vulnerability detection without extensibility consideration. To overcome the scalability and generalization limitations of existing works, we propose ESCORT, the first Deep Neural Network (DNN)-based vulnerability detection framework for Ethereum smart contracts that support lightweight transfer learning on unseen security vulnerabilities, thus is extensible and generalizable. ESCORT leverages a multi-output NN architecture that consists of two parts: (i) A common feature extractor that learns the semantics of the input contract; (ii) Multiple branch structures where each branch learns a specific vulnerability type based on features obtained from the feature extractor. Experimental results show that ESCORT achieves an average F1-score of 95% on six vulnerability types and the detection time is 0.02 seconds per contract. When extended to new vulnerability types, ESCORT yields an average F1-score of 93%. To the best of our knowledge, ESCORT is the first framework that enables transfer learning on new vulnerability types with minimal modification of the DNN model architecture and re-training overhead.}, author = {Lutz, Oliver and Chen, Huili and Fereidooni, Hossein and Sendner, Christoph and Dmitrienko, Alexandra and Sadeghi, Ahmad Reza and Koushanfar, Farinaz}, journal = {ArXiv | arXiv:2103.12607v1}, keywords = {technical-reports sss-group security cryptography}, month = {mar}, title = {ESCORT: Ethereum Smart COntRacTs Vulnerability Detection using Deep Neural Network and Transfer Learning}, year = 2021 }

A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools. M. Züfle; F. Moog; V. Lesch; C. Krupitzer; S. Kounev; in ISA Transactions: The Journal of Automation (2021).
(in press)
[ Abstract ] [ BibTeX ] [ URL ]
 
Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.
@article{zuefle2021anomalydetection, abstract = {Despite the increased sensor-based data collection in Industry 4.0, the practical use of this data is still in its infancy. In contrast, academic literature provides several approaches to detect machine failures but, in most cases, relies on simulations and vast amounts of training data. Since it is often not practical to collect such amounts of data in an industrial context, we propose an approach to detect the current production mode and machine degradation states on a comparably small data set. Our approach integrates domain knowledge about manufacturing systems into a highly generalizable end-to-end workflow ranging from raw data processing, phase segmentation, data resampling, and feature extraction to machine tool anomaly detection. The workflow applies unsupervised clustering techniques to identify the current production mode and supervised classification models for detecting the present degradation. A resampling strategy and classical machine learning models enable the workflow to handle small data sets and distinguish between normal and abnormal machine tool behavior. To the best of our knowledge, there exists no such end-to-end workflow in the literature that uses the entire machine signal as input to identify anomalies for individual tools. Our evaluation with data from a real multi-purpose machine shows that the proposed workflow detects anomalies with an average F1-score of almost 93%.}, author = {Züfle, Marwin and Moog, Felix and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, journal = {ISA Transactions: The Journal of Automation}, keywords = {Cyber-Physical_Systems descartes statistical_estimation_and_machine_learning Industry_4_0 prediction t_interdisciplinary t_journalmagazine}, note = {(in press)}, title = {A Machine Learning-based Workflow for Automatic Detection of Anomalies in Machine Tools}, year = 2021 }

Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping. N. Finke; M. Mohr; A. Lontke; M. Züfle; S. Kounev; R. Möller; in Research Challenges in Information Science (RCIS) (2021). 189–204.
[ BibTeX ]
 
@inproceedings{finke2021recommendations, author = {Finke, Nils and Mohr, Marisa and Lontke, Alexander and Z{ü}fle, Marwin and Kounev, Samuel and M{ö}ller, Ralf}, booktitle = {Research Challenges in Information Science (RCIS)}, keywords = {t_full descartes statistical_estimation_and_machine_learning prediction t_interdisciplinary}, organization = {Springer}, pages = {189–204}, title = {Recommendations for Data-Driven Degradation Estimation with Case Studies from Manufacturing and Dry-Bulk Shipping}, year = 2021 }

Establishing a cardiac training group for patients with heart failure: the “HIP-in-Würzburg” study. G. Güder; J. Wilkesmann; N. Scholz; R. Leppich; P. Düking; B. Sperlich; C. Rost; S. Frantz; C. Morbach; F. Sahiti; others; in Clinical Research in Cardiology (2021). 1–10.
[ BibTeX ]
 
@article{guder2021establishing, author = {G{ü}der, G{ü}lmisal and Wilkesmann, Joana and Scholz, Nina and Leppich, Robert and D{ü}king, Peter and Sperlich, Billy and Rost, Christian and Frantz, Stefan and Morbach, Caroline and Sahiti, Floran and others}, journal = {Clinical Research in Cardiology}, keywords = {descartes medicine t_journalmagazine}, pages = {1–10}, publisher = {Springer}, title = {Establishing a cardiac training group for patients with heart failure: the “HIP-in-W{ü}rzburg” study}, year = 2021 }

A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0. M. Züfle; J. Agne; J. Grohmann; I. Dörtoluk; S. Kounev; in Proceedings of the 21st IEEE IES International Conference on Industrial Informatics (2021).
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.
@inproceedings{ZuAgGrDoKo-INDIN2021-PdM, abstract = {Predictive maintenance is an essential aspect of the concept of Industry 4.0. In contrast to previous maintenance strategies, which plan repairs based on periodic schedules or threshold values, predictive maintenance is normally based on estimating the time-to-failure of machines. Thus, predictive maintenance enables a more efficient and effective maintenance approach. Although much research has already been done on time-to-failure prediction, most existing works provide only specialized approaches for specific machines. In most cases, these are either rotary machines (i.e., bearings) or lithium-ion batteries. To bridge the gap to a more general time-to-failure prediction, we propose a generic end-to-end predictive maintenance methodology for the time-to-failure prediction of industrial machines. Our methodology exhibits a number of novel aspects including a universally applicable method for feature extraction based on different types of sensor data, well-known feature transformation and selection techniques, adjustable target class assignment based on fault records with three different labeling strategies, and the training of multiple state-of-the-art machine learning classification models including hyperparameter optimization. We evaluated our time-to-failure prediction methodology in a real-world case study consisting of monitoring data gathered over several years from a large industrial press. The results demonstrated the effectiveness of the proposed methodology for six different time-to-failure pre-diction windows, as well as for the downscaled binary prediction of impending failures. In this case study, the multi-class feed-forward neural network model achieved the overall best results.}, author = {Züfle, Marwin and Agne, Joachim and Grohmann, Johannes and Dörtoluk, Ibrahim and Kounev, Samuel}, booktitle = {Proceedings of the 21st IEEE IES International Conference on Industrial Informatics}, keywords = {Internet_of_Things Cyber-Physical_Systems t_full descartes statistical_estimation_and_machine_learning prediction t_interdisciplinary}, month = {July}, publisher = {IEEE}, series = {INDIN'21}, title = {A Predictive Maintenance Methodology: Predicting the Time-to-Failure of Machines in Industry 4.0}, year = 2021 }

SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation. J. Grohmann; S. Eismann; A. Bauer; S. Spinner; J. Blum; N. Herbst; S. Kounev; in ACM Transactions on Autonomous and Adaptive Systems (2021). 15(2)
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.
@article{GrEiBaSpBlHeKo2021-TaaS-SARDE, abstract = {Resource demands are crucial parameters for modeling and predicting the performance of software systems. Currently, resource demand estimators are usually executed once for system analysis. However, the monitored system, as well as the resource demand itself, are subject to constant change in runtime environments. These changes additionally impact the applicability, the required parametrization as well as the resulting accuracy of individual estimation approaches. Over time, this leads to invalid or outdated estimates, which in turn negatively influence the decision-making of adaptive systems. In this article, we present SARDE, a framework for self-adaptive resource demand estimation in continuous environments. SARDE dynamically and continuously tunes, selects, and executes an ensemble of resource demand estimation approaches to adapt to changes in the environment. This creates an autonomous and unsupervised ensemble estimation technique, providing reliable resource demand estimations in dynamic environments. We evaluate SARDE using two realistic datasets. One set of different micro-benchmarks reflecting different possible system states and one dataset consisting of a continuously running application in a changing environment. Our results show that by continuously applying online optimization, selection and estimation, SARDE is able to efficiently adapt to the online trace and reduce the model error using the resulting ensemble technique.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Eismann, Simon and Bauer, Andr{é} and Spinner, Simon and Blum, Johannes and Herbst, Nikolas and Kounev, Samuel}, journal = {ACM Transactions on Autonomous and Adaptive Systems}, keywords = {descartes algorithms resource Self-adaptive learning sys:relevantfor:se-group demand performance self-tuning estimation systems optimization statistical_estimation_and_machine_learning machine prediction t_journalmagazine}, month = {jun}, number = 2, publisher = {Association for Computing Machinery}, title = {SARDE: A Framework for Continuous and Self-Adaptive Resource Demand Estimation}, volume = 15, year = 2021 }

RIP StrandHogg: A Practical StrandHogg Attack Detection Method on Android. J. Stang; A. Dmitrienko; S. Roth; in 14th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec) (2021).
[ Abstract ] [ BibTeX ]
 
StrandHogg vulnerabilities affect Android’s multitasking system and threaten up to 90% of Android platforms, which translates to millions of affected users. Existing countermeasures require modification of the OS, have usability drawbacks, or are limited to the detection of certain attack versions. In this work, we aim to develop a generic, efficient, and usability-friendly attack detection method, which does not require OS modifications and can be employed by apps installed on any vulnerable Android platform. To achieve our goal, we analyze StrandHogg attack techniques and develop two countermeasures, one using Machine Learning and the other one using ActivityCounter – a reliable attack indicator, which we could synthetically engineer. Our first approach achieves an average F1 score of 92% across all attack variations, while ActivityCounter shows superior performance and efficiently detects all attack versions without false positives. ActivityCounter is the first solution without practical limitations, which can be easily deployed in practice and protect millions of affected users.
@inproceedings{stang2021strandhogg, abstract = {StrandHogg vulnerabilities affect Android’s multitasking system and threaten up to 90% of Android platforms, which translates to millions of affected users. Existing countermeasures require modification of the OS, have usability drawbacks, or are limited to the detection of certain attack versions. In this work, we aim to develop a generic, efficient, and usability-friendly attack detection method, which does not require OS modifications and can be employed by apps installed on any vulnerable Android platform. To achieve our goal, we analyze StrandHogg attack techniques and develop two countermeasures, one using Machine Learning and the other one using ActivityCounter – a reliable attack indicator, which we could synthetically engineer. Our first approach achieves an average F1 score of 92% across all attack variations, while ActivityCounter shows superior performance and efficiently detects all attack versions without false positives. ActivityCounter is the first solution without practical limitations, which can be easily deployed in practice and protect millions of affected users.}, author = {Stang, Jasper and Dmitrienko, Alexandra and Roth, Sascha}, booktitle = {14th ACM Conference on Security and Privacy in Wireless and Mobile Networks (WiSec)}, keywords = {StrandHogg sss-group security Detection Android}, title = {RIP StrandHogg: A Practical StrandHogg Attack Detection Method on Android}, year = 2021 }

Auswirkungen der COVID-19-Pandemie auf Sportlerinnen und Sportler in Deutschland. C. Zinner; M. Matzka; R. Leppich; S. Kounev; H.-C. Holmberg; B. Sperlich; in 53. asp Jahrestagung (2021).
[ BibTeX ]
 
@article{zinner2021auswirkungen, author = {Zinner, Christoph and Matzka, Manuel and Leppich, Robert and Kounev, Samuel and Holmberg, Hans-Christian and Sperlich, Billy}, journal = {53. asp Jahrestagung}, keywords = {t_extendedabstract descartes t_interdisciplinary}, title = {Auswirkungen der COVID-19-Pandemie auf Sportlerinnen und Sportler in Deutschland}, year = 2021 }

SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications. J. Grohmann; M. Straesser; A. Chalbani; S. Eismann; Y. Arian; N. Herbst; N. Peretz; S. Kounev; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
Acceptance Rate: 29%
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing---a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90%.
@inproceedings{GrStChEiArHePeKo2021-ICPE, abstract = {Application performance management (APM) tools are useful to observe the performance properties of an application during production. However, APM is normally purely reactive, that is, it can only report about current or past performance degradation. Although some approaches capable of predictive application monitoring have been proposed, they can only report a predicted degradation but cannot explain its root-cause, making it hard to prevent the expected degradation. In this paper, we present SuanMing—a framework for predicting performance degradation of microservice applications running in cloud environments. SuanMing is able to predict future root causes for anticipated performance degradations and therefore aims at preventing performance degradations before they actually occur. We evaluate SuanMing on two realistic microservice applications, TeaStore and TrainTicket, and we show that our approach is able to predict and pinpoint performance degradations with an accuracy of over 90\%.}, address = {New York, NY, USA}, author = {Grohmann, Johannes and Straesser, Martin and Chalbani, Avi and Eismann, Simon and Arian, Yair and Herbst, Nikolas and Peretz, Noam and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {t_full descartes statistical_estimation_and_machine_learning prediction performance}, month = {April}, note = {Acceptance Rate: 29%}, organization = {ACM}, title = {SuanMing: Explainable Prediction of Performance Degradations in Microservice Applications}, year = 2021 }

Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads. J. Domaschka; S. Eismann; M. Leznik; J. Grohmann; S. Kounev; D. Seybold; in Companion of the ACM/SPEC International Conference on Performance Engineering (2021). (Vol. ICPE ’21) 175–178.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Distributed Database Management Systems (DBMS) are a crucial component of modern IT applications. Understanding their performance and non-functional properties is of paramount importance. Yet, benchmarking distributed DBMS has proven to be difficult in practice. Either, a realistic workload is often mapped to a synthetic workload without knowing if this mapping is correct or available workload traces are replayed. While the latter approach provides more realistic results, real-world traces are hard to obtain and their scope is limited in time scale and variance.We propose collecting real-world traces and then applying data generation techniques to synthesize similar realistic traces based on it. Based in this approach, we can obtain workloads for benchmarking, exhibit variability with respect to different aspects of interest while still being similar to the original traces. Varying generation parameters, we are able to support benchmarking what-if scenarios with hypothetical workloads and introduced anomalies.
@inproceedings{domaschka2021buzzy, abstract = {Distributed Database Management Systems (DBMS) are a crucial component of modern IT applications. Understanding their performance and non-functional properties is of paramount importance. Yet, benchmarking distributed DBMS has proven to be difficult in practice. Either, a realistic workload is often mapped to a synthetic workload without knowing if this mapping is correct or available workload traces are replayed. While the latter approach provides more realistic results, real-world traces are hard to obtain and their scope is limited in time scale and variance.We propose collecting real-world traces and then applying data generation techniques to synthesize similar realistic traces based on it. Based in this approach, we can obtain workloads for benchmarking, exhibit variability with respect to different aspects of interest while still being similar to the original traces. Varying generation parameters, we are able to support benchmarking what-if scenarios with hypothetical workloads and introduced anomalies.}, address = {New York, NY, USA}, author = {Domaschka, Jörg and Eismann, Simon and Leznik, Mark and Grohmann, Johannes and Kounev, Samuel and Seybold, Daniel}, booktitle = {Companion of the ACM/SPEC International Conference on Performance Engineering}, keywords = {t_visionposition descartes}, pages = {175–178}, publisher = {Association for Computing Machinery}, title = {Buzzy: Towards Realistic DBMS Benchmarking via Tailored, Representative, Synthetic Workloads}, volume = {ICPE '21}, year = 2021 }

Automatisierte Hybride Zeitreihenprognose. A. Bauer; in Ausgezeichnete Informatikdissertationen 2020, S. Hölldobler (ed.) (2021). 29–38.
[ BibTeX ]
 
@inproceedings{mci/Bauer2021, address = {Bonn}, author = {Bauer, André}, booktitle = {Ausgezeichnete Informatikdissertationen 2020}, editor = {Hölldobler, Steffen}, keywords = {chameloen cloud descartes metrics_and_benchmarking_methodologies elasticity forecasting t_bookchapter telescope}, pages = {29-38}, publisher = {Gesellschaft für Informatik e.V.}, title = {Automatisierte Hybride Zeitreihenprognose}, year = 2021 }

Performance Evaluation of a Post-Quantum Public-Key Cryptosystem. T. Prantl; D. Prantl; L. Beierlieb; L. Iffländer; A. Dmitrienko; C. Krupitzer; S. Kounev; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
[ BibTeX ]
 
@inproceedings{PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem, author = {Prantl, Thomas and Prantl, Dominik and Beierlieb, Lukas and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {t_full descartes sss-group sys:relevantfor:se-group t_workshop}, organization = {IEEE}, title = {Performance Evaluation of a Post-Quantum Public-Key Cryptosystem}, year = 2021 }

Utility-based Vehicle Routing Integrating User Preferences. V. Lesch; M. Hadry; S. Kounev; C. Krupitzer; in Proceedings of 3rd International Workshop on Pervasive Computing for Vehicular Systems (PerVehicle), 2021 (2021).
[ BibTeX ] [ Download ]
 
@inproceedings{lesch2021utility, author = {Lesch, Veronika and Hadry, Marius and Kounev, Samuel and Krupitzer, Christian}, booktitle = {Proceedings of 3rd International Workshop on Pervasive Computing for Vehicular Systems (PerVehicle), 2021}, keywords = {descartes intelligent_transportation_systems cyber-physical_systems internet_of_things t_workshop}, month = {March}, publisher = {IEEE}, title = {Utility-based Vehicle Routing Integrating User Preferences}, year = 2021 }

Towards a Group Encryption Scheme Benchmark: A View on Centralized Schemes with focus on IoT. T. Prantl; P. Ten; L. Iffländer; S. Herrnleben; A. Dmitrenko; S. Kounev; C. Krupitzer; in 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
[ BibTeX ]
 
@inproceedings{PrTeIfHeDmKo2021-ICPE-GroupEncryptionCentralizedBenchmark, author = {Prantl, Thomas and Ten, Peter and Iffländer, Lukas and Herrnleben, Stefan and Dmitrenko, Alexandra and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2021 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {Security t_short descartes sss-group Benchmark Energy\_Efficiency Group\_Encyrption\_Scheme}, month = {April}, series = {ICPE'21}, title = {Towards a Group Encryption Scheme Benchmark: A View on Centralized Schemes with focus on IoT}, year = 2021 }

Performance Impact Analysis of Securing MQTT Using TLS. T. Prantl; L. Iffländer; S. Herrnleben; S. Engel; S. Kounev; C. Krupitzer; in 2021 ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
[ BibTeX ]
 
@inproceedings{PrTeIfHeDmKo2021-ICPE-MQTT_TLS, author = {Prantl, Thomas and Iffländer, Lukas and Herrnleben, Stefan and Engel, Simon and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2021 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {Performance Security t_short descartes MQTT sss-group Pub/Sub TLS IoT}, month = {April}, series = {ICPE'21}, title = {Performance Impact Analysis of Securing MQTT Using TLS}, year = 2021 }

Libra: A Benchmark for Time Series Forecasting Methods. A. Bauer; M. Züfle; S. Eismann; J. Grohmann; N. Herbst; S. Kounev; in Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE) (2021).
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.
@inproceedings{bauer2021benchmark, abstract = {In many areas of decision making, forecasting is an essential pillar. Consequently, there are many different forecasting methods. According to the "No-Free-Lunch Theorem", there is no single forecasting method that performs best for all time series. In other words, each method has its advantages and disadvantages depending on the specific use case. Therefore, the choice of the forecasting method remains a mandatory expert task. However, expert knowledge cannot be fully automated. To establish a level playing field for evaluating the performance of time series forecasting methods in a broad setting, we propose Libra, a forecasting benchmark that automatically evaluates and ranks forecasting methods based on their performance in a diverse set of evaluation scenarios. The benchmark comprises four different use cases, each covering 100 heterogeneous time series taken from different domains. The data set was assembled from publicly available time series and was designed to exhibit much higher diversity than existing forecasting competitions. Based on this benchmark, we perform a comprehensive evaluation to compare different existing time series forecasting methods.}, address = {New York, NY, USA}, author = {Bauer, Andr{é} and Z{ü}fle, Marwin and Eismann, Simon and Grohmann, Johannes and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 12th ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {timeseries t_full descartes metrics_and_benchmarking_methodologies forecasting benchmark}, month = {April}, organization = {ACM}, title = {Libra: A Benchmark for Time Series Forecasting Methods}, year = 2021 }

A Comparison of Mechanisms for Compensating Negative Impacts of System Integration. V. Lesch; C. Krupitzer; K. Stubenrauch; N. Keil; C. Becker; S. Kounev; M. Segata; in Future Generation Computer Systems (2021). 116 117–131.
[ BibTeX ] [ DOI ] [ Download ]
 
@article{lesch2020comparison, author = {Lesch, Veronika and Krupitzer, Christian and Stubenrauch, Kevin and Keil, Nico and Becker, Christian and Kounev, Samuel and Segata, Michele}, journal = {Future Generation Computer Systems}, keywords = {descartes intelligent_transportation_systems cyber-physical_systems internet_of_things compensation system_integration t_journalmagazine platooning}, month = {March}, pages = {117-131}, title = {A Comparison of Mechanisms for Compensating Negative Impacts of System Integration}, volume = 116, year = 2021 }

Serverless Applications:Why, When, and How?. S. Eismann; S. Joel; E. van Eyk; M. Schwinger; J. Grohmann; N. Herbst; C. Abad; A. Iosup; in IEEE Software (2021). 38(1) 32–39.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Serverless computing shows good promise for efficiency and ease-of-use. Yet, there are only a few, scattered and sometimes conflicting reports on questions such as Why do so many companies adopt serverless?, When are serverless applications well suited?, and How are serverless applications currently implemented? To address these questions, we analyze 89 serverless applications from open-source projects, industrial sources, academic literature, and scientific computing—the most extensive study to date.
@article{eismann2020serverless, abstract = {Serverless computing shows good promise for efficiency and ease-of-use. Yet, there are only a few, scattered and sometimes conflicting reports on questions such as Why do so many companies adopt serverless?, When are serverless applications well suited?, and How are serverless applications currently implemented? To address these questions, we analyze 89 serverless applications from open-source projects, industrial sources, academic literature, and scientific computing—the most extensive study to date.}, author = {Eismann, Simon and Joel, Scheuner and van Eyk, Erwin and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina and Iosup, Alexandru}, journal = {IEEE Software}, keywords = {myown descartes t_journalmagazine}, number = 1, pages = {32–39}, title = {Serverless Applications:Why, When, and How?}, volume = 38, year = 2021 }

A Simulation-based Optimization Framework for Online Adaptation of Networks. S. Herrnleben; J. Grohmann; P. Rygielski; V. Lesch; C. Krupitzer; S. Kounev; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools), H. Song, D. Jiang (eds.) (2021). 513–532.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.
@inproceedings{HeGrRyLeKrKo2020-SIMUtools-Network-Online-Adaptation, abstract = {Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.}, address = {Cham}, author = {Herrnleben, Stefan and Grohmann, Johannes and Rygielski, Pitor and Lesch, Veronika and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, editor = {Song, Houbing and Jiang, Dingde}, keywords = {myown t_full descartes networking}, month = {August}, pages = {513–532}, publisher = {Springer International Publishing}, series = {SIMUtools 2020}, title = {A Simulation-based Optimization Framework for Online Adaptation of Networks}, year = 2021 }

Sizeless: Predicting the Optimal Size of Serverless Functions. S. Eismann; L. Bui; J. Grohmann; C. Abad; N. Herbst; S. Kounev; in Proceedings of the 22nd International MIDDLEWARE Conference (2021). 248–259.
Best Student Paper Award, ACM Artifacts Evaluated — Functional
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.
@inproceedings{eismann2021sizeless, abstract = {Serverless functions are an emerging cloud computing paradigm that is being rapidly adopted by both industry and academia. In this cloud computing model, the provider opaquely handles resource management tasks such as resource provisioning, deployment, and auto-scaling. The only resource management task that developers are still in charge of is selecting how much resources are allocated to each worker instance. However, selecting the optimal size of serverless functions is quite challenging, so developers often neglect it despite its significant cost and performance benefits. Existing approaches aiming to automate serverless functions resource sizing require dedicated performance tests, which are time-consuming to implement and maintain. In this paper, we introduce an approach to predict the optimal resource size of a serverless function using monitoring data from a single resource size. As our approach does not require dedicated performance tests, it enables cloud providers to implement resource sizing on a platform level and automate the last resource management task associated with serverless functions. We evaluate our approach on four different serverless applications on AWS, where it predicts the execution time of the other memory sizes based on monitoring data for a single memory size with an average prediction error of 15.3%. Based on these predictions, it selects the optimal memory size for 79.0% of the serverless functions and the secondbest memory size for 12.3% of the serverless functions, which results in an average speedup of 39.7% while also decreasing average costs by 2.6%.}, author = {Eismann, Simon and Bui, Long and Grohmann, Johannes and Abad, Cristina and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 22nd International MIDDLEWARE Conference}, keywords = {t_full descartes}, note = {Best Student Paper Award, ACM Artifacts Evaluated — Functional}, pages = {248–259}, title = {Sizeless: Predicting the Optimal Size of Serverless Functions}, year = 2021 }

2020 [ nach oben ]

Incremental Calibration of Architectural Performance Models with Parametric Dependencies. M. Mazkatli; D. Monschein; J. Grohmann; A. Koziolek; in 2020 IEEE International Conference on Software Architecture (ICSA 2020) (2020). 23–34.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Architecture-based Performance Prediction (AbPP) allows evaluation of the performance of systems and to answer what-if questions without measurements for all alternatives. A difficulty when creating models is that Performance Model Parameters (PMPs, such as resource demands, loop iteration numbers and branch probabilities) depend on various influencing factors like input data, used hardware and the applied workload. To enable a broad range of what-if questions, Performance Models (PMs) need to have predictive power beyond what has been measured to calibrate the models. Thus, PMPs need to be parametrized over the influencing factors that may vary. Existing approaches allow for the estimation of parametrized PMPs by measuring the complete system. Thus, they are too costly to be applied frequently, up to after each code change. They do not keep also manual changes to the model when recalibrating. In this work, we present the Continuous Integration of Performance Models (CIPM), which incrementally extracts and calibrates the performance model, including parametric dependencies. CIPM responds to source code changes by updating the PM and adaptively instrumenting the changed parts. To allow AbPP, CIPM estimates the parametrized PMPs using the measurements (generated by performance tests or executing the system in production) and statistical analysis, e.g., regression analysis and decision trees. Additionally, our approach responds to production changes (e.g., load or deployment changes) and calibrates the usage and deployment parts of PMs accordingly. For the evaluation, we used two case studies. Evaluation results show that we were able to calibrate the PM incrementally and accurately.
@inproceedings{MaMoGrKo2020-ICSA, abstract = {Architecture-based Performance Prediction (AbPP) allows evaluation of the performance of systems and to answer what-if questions without measurements for all alternatives. A difficulty when creating models is that Performance Model Parameters (PMPs, such as resource demands, loop iteration numbers and branch probabilities) depend on various influencing factors like input data, used hardware and the applied workload. To enable a broad range of what-if questions, Performance Models (PMs) need to have predictive power beyond what has been measured to calibrate the models. Thus, PMPs need to be parametrized over the influencing factors that may vary. Existing approaches allow for the estimation of parametrized PMPs by measuring the complete system. Thus, they are too costly to be applied frequently, up to after each code change. They do not keep also manual changes to the model when recalibrating. In this work, we present the Continuous Integration of Performance Models (CIPM), which incrementally extracts and calibrates the performance model, including parametric dependencies. CIPM responds to source code changes by updating the PM and adaptively instrumenting the changed parts. To allow AbPP, CIPM estimates the parametrized PMPs using the measurements (generated by performance tests or executing the system in production) and statistical analysis, e.g., regression analysis and decision trees. Additionally, our approach responds to production changes (e.g., load or deployment changes) and calibrates the usage and deployment parts of PMs accordingly. For the evaluation, we used two case studies. Evaluation results show that we were able to calibrate the PM incrementally and accurately.}, author = {Mazkatli, Manar and Monschein, David and Grohmann, Johannes and Koziolek, Anne}, booktitle = {2020 IEEE International Conference on Software Architecture (ICSA 2020)}, keywords = {myown modeling t_full descartes statistical_estimation_and_machine_learning Automated_model_learning prediction performance}, month = {March}, pages = {23–34}, publisher = {IEEE}, title = {Incremental Calibration of Architectural Performance Models with Parametric Dependencies}, year = 2020 }

Optimizing Parametric Dependencies for Incremental Performance Model Extraction. S. Voneva; M. Mazkatli; J. Grohmann; A. Koziolek; in Companion of the 14th European Conference Software Architecture (ECSA 2020), H. Muccini, P. Avgeriou, B. Buhnova, J. Cámara, M. Caporuscio, M. Franzago, A. Koziolek, P. Scandurra, C. Trubiani, D. Weyns, U. Zdun (eds.) (2020). (Vol. 1269) 228–240.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Model-based performance prediction in agile software development promises to evaluate design alternatives and to reduce the cost of performance tests. To minimize the differences between a real software and its performance model, parametric dependencies are introduced. They express how the performance model parameters (such as loop iteration count, branch transition probabilities, resource demands, and external service call arguments) depend on impacting factors like the input data. The approaches that perform model-based performance prediction in agile software development have two major shortcomings: they are either costly because they do not update the performance models automatically after each commit, or do not consider more complex parametric dependencies than linear. This work extends an approach for continuous integration of performance model during agile development. Our extension aims to optimize the learning of parametric dependencies with a genetic programming algorithm to be able to detect non-linear dependencies. The case study results show that using genetic programming enables detecting more complex dependencies and improves the accuracy of the updated performance model.
@inproceedings{VoMaGrKo-ECSA-Dependencies, abstract = {Model-based performance prediction in agile software development promises to evaluate design alternatives and to reduce the cost of performance tests. To minimize the differences between a real software and its performance model, parametric dependencies are introduced. They express how the performance model parameters (such as loop iteration count, branch transition probabilities, resource demands, and external service call arguments) depend on impacting factors like the input data. The approaches that perform model-based performance prediction in agile software development have two major shortcomings: they are either costly because they do not update the performance models automatically after each commit, or do not consider more complex parametric dependencies than linear. This work extends an approach for continuous integration of performance model during agile development. Our extension aims to optimize the learning of parametric dependencies with a genetic programming algorithm to be able to detect non-linear dependencies. The case study results show that using genetic programming enables detecting more complex dependencies and improves the accuracy of the updated performance model.}, address = {Cham}, author = {Voneva, Sonya and Mazkatli, Manar and Grohmann, Johannes and Koziolek, Anne}, booktitle = {Companion of the 14th European Conference Software Architecture ({ECSA} 2020)}, editor = {Muccini, Henry and Avgeriou, Paris and Buhnova, Barbora and C{{á}}mara, Javier and Caporuscio, Mauro and Franzago, Mirco and Koziolek, Anne and Scandurra, Patrizia and Trubiani, Catia and Weyns, Danny and Zdun, Uwe}, keywords = {myown descartes statistical_estimation_and_machine_learning parametric_dependencies prediction performance t_workshop}, month = {September}, pages = {228–240}, publisher = {Springer}, series = {Communications in Computer and Information Science}, title = {Optimizing Parametric Dependencies for Incremental Performance Model Extraction}, volume = 1269, year = 2020 }

SIMPL: Secure IoT Management Platform. T. Prantl; A. E. Ben Yahya; A. Dmitrienko; S. Kounev; F. Lipp; D. Hock; C. Rathfelder; M. Hofherr; in ITG Workshop on IT Security (ITSec) (2020).
[ Abstract ] [ BibTeX ] [ Download ]
 
The proliferation of IoT devices is increasing at a fast pace, whether for private or business use. However,despite their growing popularity, their safe operation is often not guaranteed. To tackle the security challenges of modern IoT environments, the German Federal Ministry of Education and Research is funding the SIMPL project in order to support research in this area. The contribution of this paper is to introduce the main goal, objectives, and key features of SIMPL
@inproceedings{PrYaDmKoLiHoRaHo2020-ITSec-SIMPL, abstract = {The proliferation of IoT devices is increasing at a fast pace, whether for private or business use. However,despite their growing popularity, their safe operation is often not guaranteed. To tackle the security challenges of modern IoT environments, the German Federal Ministry of Education and Research is funding the SIMPL project in order to support research in this area. The contribution of this paper is to introduce the main goal, objectives, and key features of SIMPL}, author = {Prantl, Thomas and Ben Yahya, Ala Eddine and Dmitrienko, Alexandra and Kounev, Samuel and Lipp, Fabian and Hock, David and Rathfelder, Christoph and Hofherr, Martin}, booktitle = {ITG Workshop on IT Security (ITSec)}, keywords = {SIMPL myown descartes internet_of_things t_workshop}, title = {SIMPL: Secure IoT Management Platform}, year = 2020 }

Towards a Self-Aware Prediction of Critical States. M. Züfle; in Organic Computing: Doctoral Dissertation Colloquium 2020, S. Tomforde, C. Krupitzer (eds.) (2020).
(to appear)
[ BibTeX ]
 
@incollection{zufle2020towards, author = {Z{ü}fle, Marwin}, booktitle = {Organic Computing: Doctoral Dissertation Colloquium 2020}, editor = {Tomforde, Sven and Krupitzer, Christian}, keywords = {descartes Self-aware-computing statistical_estimation_and_machine_learning prediction online_monitoring_and_forecasting t_bookchapter}, month = {jul}, note = {(to appear)}, title = {Towards a Self-Aware Prediction of Critical States}, year = 2020 }

Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS. J. Grohmann; D. Seybold; S. Eismann; M. Leznik; S. Kounev; J. Domaschka; in 2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020). 1–8.
Acceptance Rate: 27%
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.
@inproceedings{GrSeEiLeKoDo2020-MASCOTS-DBMSPerformance, abstract = {Correctly configuring a distributed database management system (DBMS) deployed in a cloud environment for maximizing performance poses many challenges to operators. Even if the entire configuration spectrum could be measured directly, which is often infeasible due to the multitude of parameters, single measurements are subject to random variations and need to be repeated multiple times. In this work, we propose Baloo, a framework for systematically measuring and modeling different performance-relevant configurations of distributed DBMS in cloud environments. Baloo dynamically estimates the required number of configurations, as well as the number of required measurement repetitions per configuration based on a desired target accuracy. We evaluate Baloo based on a data set consisting of 900 DBMS configuration measurements conducted in our private cloud setup. Our evaluation shows that the highly configurable framework is able to achieve a prediction error of up to 12% while saving 80% of the measurement effort. We also publish all code and the acquired data set to foster future research.}, author = {Grohmann, Johannes and Seybold, Daniel and Eismann, Simon and Leznik, Mark and Kounev, Samuel and Domaschka, Jörg}, booktitle = {2020 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {myown t_full performance_modeling descartes statistical_estimation_and_machine_learning prediction performance automated_model_learning}, month = {November}, note = {Acceptance Rate: 27%}, pages = {1–8}, publisher = {IEEE}, series = {MASCOTS '20}, title = {Baloo: Measuring and Modeling the Performance Configurations of Distributed DBMS}, year = 2020 }

An Overview of Design Patterns for Self-Adaptive Systems in the Context of the Internet of Things. C. Krupitzer; T. Temizer; T. Prantl; C. Raibulet; in IEEE Access (2020).
[ Abstract ] [ BibTeX ] [ Download ]
 
The Internet of Things (IoT) requires the integration of all available, highly specialized, and heterogeneous devices, ranging from embedded sensor nodes to servers in the cloud. The self-adaptive research domain provides adaptive capabilities that can support the integration in IoT systems. However, developing such systems is a challenging, error-prone, and time-consuming task. In this context, design patterns propose already used and optimized solutions to specific problems in various contexts. Applying design patterns might help to reuse existing knowledge about similar development issues. However, so far, there is a lack of taxonomies on design patterns for self-adaptive systems. To tackle this issue, in this paper, we provide a taxonomy on design patterns for self-adaptive systems that can be transferred to support adaptivity in IoT systems. Besides describing the taxonomy and the design patterns, we discuss their applicability in an Industrial IoT case study.
@article{ieeeaccess2020:sasdesignpatterns, abstract = {The Internet of Things (IoT) requires the integration of all available, highly specialized, and heterogeneous devices, ranging from embedded sensor nodes to servers in the cloud. The self-adaptive research domain provides adaptive capabilities that can support the integration in IoT systems. However, developing such systems is a challenging, error-prone, and time-consuming task. In this context, design patterns propose already used and optimized solutions to specific problems in various contexts. Applying design patterns might help to reuse existing knowledge about similar development issues. However, so far, there is a lack of taxonomies on design patterns for self-adaptive systems. To tackle this issue, in this paper, we provide a taxonomy on design patterns for self-adaptive systems that can be transferred to support adaptivity in IoT systems. Besides describing the taxonomy and the design patterns, we discuss their applicability in an Industrial IoT case study.}, author = {Krupitzer, Christian and Temizer, Timur and Prantl, Thomas and Raibulet, Claudia}, journal = {IEEE Access}, keywords = {descartes cyber-physical_systems internet_of_things t_journalmagazine sys:relevantfor:se-group self-adaptive-systems}, title = {An Overview of Design Patterns for Self-Adaptive Systems in the Context of the Internet of Things}, year = 2020 }

Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data. F. Spiess; J. Friesslich; K. Tobias; S. Kounev; N. Strobel; in International Journal of Mechanical Engineering and Robotics Research (IJMERR), (P. R. (Chunhui) Yang, ed.) (2020). 9(12) 1532–1540.
[ Abstract ] [ BibTeX ] [ URL ]
 
This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.
@article{yang2020survey, abstract = {This paper presents an experimental evaluation and comparison of selected Visual Odometry (VO) and Visual-SLAM (V-SLAM) algorithms for indoor mobile robot navigation supported by the Robot Operating System (ROS). The focus is on algorithms involving RGB-D cameras. Since RGB-D cameras integrate color and depth information, they output coherent measurement data and facilitate an efficient processing pipeline. The various underlying methods of vision-based algorithms are described and evaluated on two datasets covering different indoor situations as well as various lighting and movement conditions. In general, V-SLAM algorithms yielded better results. They were superior with respect to handling drift, in particular when loop closures were involved. However, the results confirmed that VO algorithms could outperform V-SLAM methods under certain circumstances. This happened when there was a very good match between an algorithm’s design objectives and the situation at hand. While the experiments showed that there is no single best algorithm for every scenario, ORB-SLAM2 is recommended as a robust stand-alone RGB-D based localization method available under ROS. Furthermore, we observed that the position estimation error could be reduced by around 67% on average when combining vision-based position estimates with sensor data obtained from wheel odometry and an inertial measurement unit (IMU), respectively. This clearly demonstrates the potential of sensor fusion techniques. The best results in case of sensor fusion were obtained with RGB-DSLAMv2.}, author = {Spiess, Florian and Friesslich, Jonas and Tobias, Kaupp and Kounev, Samuel and Strobel, Norbert}, editor = {Yang, Prof Richard (Chunhui)}, journal = {International Journal of Mechanical Engineering and Robotics Research (IJMERR)}, keywords = {comparison descartes robotics survey t_interdisciplinary t_journalmagazine rgb}, month = {dec}, number = 12, pages = {1532-1540}, title = {Survey and Experimental Comparison of RGB-D Indoor Robot Navigation Methods Supported by ROS and Their Expansion via Fusion with Wheel Odometry and IMU Data}, volume = 9, year = 2020 }

Elasticity of Cloud Platforms. N. Herbst; A. Bauer; S. Kounev; in Systems Benchmarking: For Scientists and Engineers, S. Kounev, K.-D. Lange, J. von Kistowski (eds.) (2020). 319–340.
[ BibTeX ]
 
@inproceedings{bauer2020bookelasticity, author = {Herbst, Nikolas and Bauer, Andr{é} and Kounev, Samuel}, booktitle = {{Systems Benchmarking: For Scientists and Engineers}}, editor = {Kounev, Samuel and Lange, Klaus-Dieter and von Kistowski, J{ó}akim}, keywords = {cloud descartes metrics_and_benchmarking_methodologies elasticity t_bookchapter}, month = 1, organization = {Springer}, pages = {319-340}, title = {{Elasticity of Cloud Platforms}}, year = 2020 }

Detection of Software Vulnerabilities in Smart Contracts using Deep Learning. O. Lutz; Thesis; University of Würzburg. (2020, October).
[ Abstract ] [ BibTeX ]
 
Der Missbrauch von Programmierfehlern in Kryptowährungsplattformen kann zu großen wirtschaftlichen Schäden fuhren. Deshalb wird in dieser Arbeit analysiert, wie Deep-Learning genutzt werden kann, um Schwachstellen in Ethereum Smart Contracts aufzudecken. Als Ergebnis ist ein Framework entwickelt worden, das sich um die Datengewinnung, Vorklassifizierung und Modellbildung kummert. Mit diesem Framework können Deep-Learning-Modelle entwickelt, trainiert und evaluiert werden, um einen KI-Scanner zu erstellen. Selbst bei einfachen neuronalen Netz-Architekturen erreicht der KI-Scanner, der acht verschiedene Arten von Schwachstellen erkennen kann, eine Genauigkeit von 98%. Dieser KI-Scanner kann zur weiteren Verwendung eingesetzt werden. Er kann jedoch auch um Schwachstellentypen erweitert werden, um ein noch mächtigeres Tool zu bilden.
@mastersthesis{lutz2020detection, abstract = {Der Missbrauch von Programmierfehlern in Kryptowährungsplattformen kann zu großen wirtschaftlichen Schäden fuhren. Deshalb wird in dieser Arbeit analysiert, wie Deep-Learning genutzt werden kann, um Schwachstellen in Ethereum Smart Contracts aufzudecken. Als Ergebnis ist ein Framework entwickelt worden, das sich um die Datengewinnung, Vorklassifizierung und Modellbildung kummert. Mit diesem Framework können Deep-Learning-Modelle entwickelt, trainiert und evaluiert werden, um einen KI-Scanner zu erstellen. Selbst bei einfachen neuronalen Netz-Architekturen erreicht der KI-Scanner, der acht verschiedene Arten von Schwachstellen erkennen kann, eine Genauigkeit von 98%. Dieser KI-Scanner kann zur weiteren Verwendung eingesetzt werden. Er kann jedoch auch um Schwachstellentypen erweitert werden, um ein noch mächtigeres Tool zu bilden.}, author = {Lutz, Oliver}, keywords = {deep-learning thesis_supervised_by_SSS_member sss-group smart-contracts}, month = {October}, school = {University of W{ü}rzburg}, title = {Detection of Software Vulnerabilities in Smart Contracts using Deep Learning}, type = {Master Thesis}, year = 2020 }

Testbed for Security Testing of Smart Contracts. L. Denk; Thesis; University of Würzburg. (2020, November).
[ Abstract ] [ BibTeX ]
 
Ethereum, one of the most popular decentralized blockchain-based platforms [1], manages cryptocurrency worth more than 40 billion US Dollars [2]. A lot of the money is controlled by autonomous programs, so called smart contracts. They are executed on the platform and everyone can call their functions; vulnerabilities are therefore easily exploited. The immutable nature of the blockchain sharpens the problem even further by prohibiting belated bug fixes. In fact, there were several cases where attackers were able to steal cryptocurrency worth several millions of US Dollar [3]. This emphasises the need of soundly testing a smart contract before deployment. Unfortunately, existing security analysing tools each cover a different subsets of vulnerabilities [4]. Moreover, even when they test for the same security issues, they often find different results. A solid test run should therefore contain several tools, which makes the whole procedure very laborious. For these reasons, we provide a testbed which runs on a virtual machine (VM) with several smart contract security analysing tools integrated. It does not only offer a command line- but also an intuitive web interface. Both interfaces allow the user to analyse his contract with all the underlying tools in a single step. Additionally, our testbed provides a detailed report for each of the tools’ test runs as well as an overview of the overall security issues the different tools found. Our evaluation shows that our testbed identifies significantly more potential security issues than each of the compared smart contract analysing tools individually does. Indeed, we found 92% of our analysed contracts as potentially being vulnerable, while even the most sensitive tool only flagged 76% of its successfully analysed contracts. Furthermore, we observed that occasionally, two tools testing for the same vulnerability, contradict themselves in more than 81% of the contracts successfully analysed by both tools
@mastersthesis{denk2020testbed, abstract = {Ethereum, one of the most popular decentralized blockchain-based platforms [1], manages cryptocurrency worth more than 40 billion US Dollars [2]. A lot of the money is controlled by autonomous programs, so called smart contracts. They are executed on the platform and everyone can call their functions; vulnerabilities are therefore easily exploited. The immutable nature of the blockchain sharpens the problem even further by prohibiting belated bug fixes. In fact, there were several cases where attackers were able to steal cryptocurrency worth several millions of US Dollar [3]. This emphasises the need of soundly testing a smart contract before deployment. Unfortunately, existing security analysing tools each cover a different subsets of vulnerabilities [4]. Moreover, even when they test for the same security issues, they often find different results. A solid test run should therefore contain several tools, which makes the whole procedure very laborious. For these reasons, we provide a testbed which runs on a virtual machine (VM) with several smart contract security analysing tools integrated. It does not only offer a command line- but also an intuitive web interface. Both interfaces allow the user to analyse his contract with all the underlying tools in a single step. Additionally, our testbed provides a detailed report for each of the tools’ test runs as well as an overview of the overall security issues the different tools found. Our evaluation shows that our testbed identifies significantly more potential security issues than each of the compared smart contract analysing tools individually does. Indeed, we found 92% of our analysed contracts as potentially being vulnerable, while even the most sensitive tool only flagged 76% of its successfully analysed contracts. Furthermore, we observed that occasionally, two tools testing for the same vulnerability, contradict themselves in more than 81% of the contracts successfully analysed by both tools}, author = {Denk, Lukas}, keywords = {thesis_supervised_by_SSS_member sss-group smart-contracts security secure}, month = {November}, school = {University of W{ü}rzburg}, title = {Testbed for Security Testing of Smart Contracts}, type = {Bachelor Thesis}, year = 2020 }

Systems Benchmarking S. Kounev; K.-D. Lange; J. von Kistowski; (2020). (1st ed.) Springer International Publishing.
[ BibTeX ]
 
@book{KoLaKi-2020-SystemsBenchmarking, author = {Kounev, Samuel and Lange, Klaus-Dieter and von Kistowski, Jóakim}, edition = 1, keywords = {descartes t_book}, publisher = {Springer International Publishing}, title = {{Systems Benchmarking}}, year = 2020 }

Evaluating the Privacy of Contact Discovery. C. Sendner; Thesis; University of Würzburg. (2020, July).
[ Abstract ] [ BibTeX ]
 
Currently deployed contact discovery of mobile messengers is based on the transmission of phone numbers to the service provider. This information is private and anonymized by hashing them. In this work, we show, that this anonymization is pseudo-anonymous and can easily be broken by an attacker. For that, we develop two hash reversal techniques: one using brute-force approach and another one using look-up databases. We provide generic architectures for each of the ap- proaches. Additionally, we provide and compare two instantiations for each. Furthermore, we evaluate and compare them to the third approach based on rainbow tables. The evaluation shows near instant lookup-times of under 0.1 ms using in-memory lookup databases, this approach is however costly in terms of memory – it would require over 10 TB RAM, which would be difficult to obtain. Our brute-force approach shows an astonishing performance, being able to reverse any mobile number in under 100 seconds using consumer-level hardware. The rainbow tables produce lookup-times of 4.5 minutes with a success rate of over 99.99%. The results of our evaluation demonstrate, that hash reversals of mobile phone numbers are practical and near instant. Thus, an attacker can easily reverse hash digests of mobile phone numbers and de-anonymize personally identifiable information – like phone numbers transmitted to the service provider of mobile messenger apps.
@mastersthesis{sendner2020contactdiscovery, abstract = {Currently deployed contact discovery of mobile messengers is based on the transmission of phone numbers to the service provider. This information is private and anonymized by hashing them. In this work, we show, that this anonymization is pseudo-anonymous and can easily be broken by an attacker. For that, we develop two hash reversal techniques: one using brute-force approach and another one using look-up databases. We provide generic architectures for each of the ap- proaches. Additionally, we provide and compare two instantiations for each. Furthermore, we evaluate and compare them to the third approach based on rainbow tables. The evaluation shows near instant lookup-times of under 0.1 ms using in-memory lookup databases, this approach is however costly in terms of memory – it would require over 10 TB RAM, which would be difficult to obtain. Our brute-force approach shows an astonishing performance, being able to reverse any mobile number in under 100 seconds using consumer-level hardware. The rainbow tables produce lookup-times of 4.5 minutes with a success rate of over 99.99%. The results of our evaluation demonstrate, that hash reversals of mobile phone numbers are practical and near instant. Thus, an attacker can easily reverse hash digests of mobile phone numbers and de-anonymize personally identifiable information – like phone numbers transmitted to the service provider of mobile messenger apps.}, author = {Sendner, Christoph}, keywords = {thesis thesis_supervised_by_sss_member thesis_supervised_by_SSS_member sss-group}, month = {July}, school = {University of W{ü}rzburg}, title = {Evaluating the Privacy of Contact Discovery}, type = {Master Thesis}, year = 2020 }

Predicting the Costs of Serverless Workflows. S. Eismann; J. Grohmann; E. van Eyk; N. Herbst; S. Kounev; in Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) (2020). 265–276.
Acceptance Rate: 23.4% (15/64)
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.
@inproceedings{EiGrEyHeKo2020-ICPE-ServerlessWorkflows, abstract = {Function-as-a-Service (FaaS) platforms enable users to run arbitrary functions without being concerned about operational issues, while only paying for the consumed resources. Individual functions are often composed into workflows for complex tasks. However, the pay-per-use model and nontransparent reporting by cloud providers make it challenging to estimate the expected cost of a workflow, which prevents informed business decisions. Existing cost-estimation approaches assume a static response time for the serverless functions, without taking input parameters into account. In this paper, we propose a methodology for the cost prediction of serverless workflows consisting of input-parameter sensitive function models and a monte-carlo simulation of an abstract workflow model. Our approach enables workflow designers to predict, compare, and optimize the expected costs and performance of a planned workflow, which currently requires time-intensive experimentation. In our evaluation, we show that our approach can predict the response time and output parameters of a function based on its input parameters with an accuracy of 96.1%. In a case study with two audio-processing workflows, our approach predicts the costs of the two workflows with an accuracy of 96.2%.}, address = {New York, NY, USA}, author = {Eismann, Simon and Grohmann, Johannes and van Eyk, Erwin and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {t_full descartes se2}, month = {April}, note = {{Acceptance Rate: 23.4% (15/64)}}, pages = {265–276}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {Predicting the Costs of Serverless Workflows}, year = 2020 }

All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers. C. Hagen; C. Weinert; C. Sendner; A. Dmitrienko; T. Schneider; in Cryptology ePrint Archive, Report 2020/1119 (2020).
[ BibTeX ] [ URL ]
 
@article{cryptoeprint:2020:1119, author = {Hagen, Christoph and Weinert, Christian and Sendner, Christoph and Dmitrienko, Alexandra and Schneider, Thomas}, journal = {Cryptology ePrint Archive, Report 2020/1119}, keywords = {technical-reports sss-group security}, title = {All the Numbers are US: Large-scale Abuse of Contact Discovery in Mobile Messengers}, year = 2020 }

Predicting Performance Degradations of Black-box Microservice Applications M. Straesser; (2020).
[ BibTeX ] [ URL ] [ Download ]
 
@presentation{straesser2020talk, author = {Straesser, Martin}, keywords = {t_talk descartes microservice performance_prediction sys:relevantfor:se-group}, title = {Predicting Performance Degradations of Black-box Microservice Applications}, year = 2020 }

A Framework for Time Series Preprocessing and History-based Forecasting Method Recommendation. M. Züfle; S. Kounev; in Proceedings of the 15th Conference on Computer Science and Information Systems (FedCSIS): Data Mining Competition of the International Symposium on Advanced Artificial Intelligence in Applications (2020).
[ BibTeX ] [ Download ]
 
@inproceedings{ZuKo-AAIA2020-Framework, author = {Z{ü}fle, Marwin and Kounev, Samuel}, booktitle = {Proceedings of the 15th Conference on Computer Science and Information Systems (FedCSIS): Data Mining Competition of the International Symposium on Advanced Artificial Intelligence in Applications}, keywords = {t_short descartes statistical_estimation_and_machine_learning prediction online_monitoring_and_forecasting}, month = {September}, title = {A Framework for Time Series Preprocessing and History-based Forecasting Method Recommendation}, year = 2020 }

REACT: A Model-Based Runtime Environment for Adapting Communication Systems. M. Pfannemüller; M. Breitbach; C. Krupitzer; M. Weckesser; C. Becker; B. Schmerl; A. Schürr; in Proceedings of the IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS 2020) (2020).
[ Abstract ] [ BibTeX ] [ Download ]
 
Trends such as the Internet of Things or edge computing lead to a growing number of networked devices. Hence, it is becoming increasingly important to manage communication systems at runtime. Adding self-adaptive capabilities is one approach to reduce administrative effort and cope with changing execution contexts. Existing frameworks for building self-adaptive software can help to reduce development effort in general. Yet, they are neither tailored towards the use in communication systems nor easily usable without profound knowledge in self-adaptive systems development. In this paper, we propose REACT, a reusable, model-based runtime environment to complement communication systems with adaptive behavior. It addresses the heterogeneity and distribution aspects of networks and reduces development effort. REACT empowers developers of communication systems to add adaptive behavior without having experience in self-adaptive systems development. We show the effectiveness and efficiency of our prototype in an experimental evaluation based on two distinct use cases from the communication systems domain: cloud resource management and software-defined networking. The first use case includes a comparison with Rainbow, which represents a state-of-the-art model-based framework for building self-adaptive systems. The second use case applies REACT in a sophisticated, real-world communication system scenario.
@conference{acsos2020:react, abstract = {Trends such as the Internet of Things or edge computing lead to a growing number of networked devices. Hence, it is becoming increasingly important to manage communication systems at runtime. Adding self-adaptive capabilities is one approach to reduce administrative effort and cope with changing execution contexts. Existing frameworks for building self-adaptive software can help to reduce development effort in general. Yet, they are neither tailored towards the use in communication systems nor easily usable without profound knowledge in self-adaptive systems development. In this paper, we propose REACT, a reusable, model-based runtime environment to complement communication systems with adaptive behavior. It addresses the heterogeneity and distribution aspects of networks and reduces development effort. REACT empowers developers of communication systems to add adaptive behavior without having experience in self-adaptive systems development. We show the effectiveness and efficiency of our prototype in an experimental evaluation based on two distinct use cases from the communication systems domain: cloud resource management and software-defined networking. The first use case includes a comparison with Rainbow, which represents a state-of-the-art model-based framework for building self-adaptive systems. The second use case applies REACT in a sophisticated, real-world communication system scenario.}, author = {Pfannemüller, Martin and Breitbach, Martin and Krupitzer, Christian and Weckesser, Markus and Becker, Christian and Schmerl, Bradley and Schürr, Andy}, booktitle = {Proceedings of the IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS 2020)}, keywords = {t_full descartes cyber-physical_systems internet_of_things sys:relevantfor:se-group self-adaptive-systems}, title = {REACT: A Model-Based Runtime Environment for Adapting Communication Systems}, year = 2020 }

Strategies for the Security Assessment of IoT Devices by Certification Authorities. M. A. Finke; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2020, May).
[ Abstract ] [ BibTeX ]
 
The Internet of Things (IoT) is a global infrastructure that interconnects physical and virtual things based on evolving information and communication technologies. Along with the rise of the IoT, multiple unprecedented forms of security issues and attacks have emerged. While their limited resources disallow the utilization of computing-intensive security measures, the services of IoT devices represent an attractive target for vulnerability exploitation. Certification Authorities (CAs) have responded to the trend and offer security assessments and certification for IoT devices. To ensure qualitative and fair certification, security testing requires well-defined strategies. This bachelor thesis analyzes the existing IoT security assessment models and proposes a novel, CA oriented Testing Guide Model (TGM) for standardized and reproducible assessments. The TGM describes a complete security assessment approach for application in IoT device security certification procedures of CAs. As part of the bachelor thesis, the TGM is specified, implemented, and its applicability evaluated with the help of existing IoT devices. CAs account for the security of certified products. The certification of devices that (subsequently) prove to be insecure can lead to a consumer's loss of trust in the CA. Therefore, the decision for issuing certificates must be comprehensible and made carefully. For the risk estimation of security issues and vulnerabilities, it is common to make use of numerical scoring systems. Well-defined scoring systems allow precise and reproducible estimations. In addition to the introduction of the TGM, this bachelor thesis analyzes the scoring systems established in the field of IT security in regard to their applicability for the scoring of IoT related vulnerabilities, complete IoT devices, and for the specific requirements of CAs. This thesis uncovers incompatibilities and insufficiencies in the existing systems that disallow application for the scoring of IoT devices and for application by CAs. To counter these issues, the thesis introduces a novel Security Scoring System (SSS) for rating IoT devices based on the risks of their vulnerabilities, services, and operational contexts. The SSS is integrated into the TGM, implemented in software, and evaluated with a comparison to the existing scoring systems. To summarize, the contributions of this bachelor thesis are two novel models for the security assessment of devices of the IoT: the TGM and the SSS. The models are intended to find application for CAs and both standardize and improve the security testing and assessment procedures.
@mastersthesis{finke2020strategies, abstract = {The Internet of Things (IoT) is a global infrastructure that interconnects physical and virtual things based on evolving information and communication technologies. Along with the rise of the IoT, multiple unprecedented forms of security issues and attacks have emerged. While their limited resources disallow the utilization of computing-intensive security measures, the services of IoT devices represent an attractive target for vulnerability exploitation. Certification Authorities (CAs) have responded to the trend and offer security assessments and certification for IoT devices. To ensure qualitative and fair certification, security testing requires well-defined strategies. This bachelor thesis analyzes the existing IoT security assessment models and proposes a novel, CA oriented Testing Guide Model (TGM) for standardized and reproducible assessments. The TGM describes a complete security assessment approach for application in IoT device security certification procedures of CAs. As part of the bachelor thesis, the TGM is specified, implemented, and its applicability evaluated with the help of existing IoT devices. CAs account for the security of certified products. The certification of devices that (subsequently) prove to be insecure can lead to a consumer's loss of trust in the CA. Therefore, the decision for issuing certificates must be comprehensible and made carefully. For the risk estimation of security issues and vulnerabilities, it is common to make use of numerical scoring systems. Well-defined scoring systems allow precise and reproducible estimations. In addition to the introduction of the TGM, this bachelor thesis analyzes the scoring systems established in the field of IT security in regard to their applicability for the scoring of IoT related vulnerabilities, complete IoT devices, and for the specific requirements of CAs. This thesis uncovers incompatibilities and insufficiencies in the existing systems that disallow application for the scoring of IoT devices and for application by CAs. To counter these issues, the thesis introduces a novel Security Scoring System (SSS) for rating IoT devices based on the risks of their vulnerabilities, services, and operational contexts. The SSS is integrated into the TGM, implemented in software, and evaluated with a comparison to the existing scoring systems. To summarize, the contributions of this bachelor thesis are two novel models for the security assessment of devices of the IoT: the TGM and the SSS. The models are intended to find application for CAs and both standardize and improve the security testing and assessment procedures.}, address = {Am Hubland, Informatikgebäude, 97074 Würzburg, Germany}, author = {Finke, Moritz Anton}, keywords = {Security thesis_supervised_by_SSS_member sss-group IoT}, month = {may}, school = {University of Würzburg}, title = {Strategies for the Security Assessment of IoT Devices by Certification Authorities}, type = {Bachelor Thesis}, year = 2020 }

LegIoT: Ledgered Trust Management Platform for IoT. J. Neureither; A. Dmitrienko; D. Koisser; F. Brasser; A.-R. Sadeghi; in European Symposium on Research in Computer Security (ESORICS) (2020).
[ Abstract ] [ BibTeX ]
 
We investigate and address the currently unsolved problem of trust establishment in large-scale Internet of Things (IoT) networks where heterogeneous devices and mutually mistrusting stakeholders are involved. We design, prototype and evaluate LegIoT, a novel, probabilistic trust management system that enables secure, dynamic and flexible(yet inexpensive) trust relationships in large IoT networks. The core component of LegIoT is a novel graph-based scheme that allows network devices (graph nodes) to re-use the already existing trust associations(graph edges) very efficiently; thus, significantly reducing the number of individually conducted trust assessments. Since no central trusted third party exists, LegIoT leverages Distributed Ledger Technology(DLT) to create and manage the trust relation graph in a decentralized manner.The trust assessment among devices can be instantiated by any appropriate assessment technique, for which we focus on remote attestation(integrity verification) in this paper. We prototyped LegIoT for Hyper-ledger Sawtooth and demonstrated through evaluation that the number of trust assessments in the network can be significantly reduced – e.g.,by a factor of 20 for a network of 400 nodes and factor 5 for 1000 nodes.
@inproceedings{neureither2020legiot, abstract = {We investigate and address the currently unsolved problem of trust establishment in large-scale Internet of Things (IoT) networks where heterogeneous devices and mutually mistrusting stakeholders are involved. We design, prototype and evaluate LegIoT, a novel, probabilistic trust management system that enables secure, dynamic and flexible(yet inexpensive) trust relationships in large IoT networks. The core component of LegIoT is a novel graph-based scheme that allows network devices (graph nodes) to re-use the already existing trust associations(graph edges) very efficiently; thus, significantly reducing the number of individually conducted trust assessments. Since no central trusted third party exists, LegIoT leverages Distributed Ledger Technology(DLT) to create and manage the trust relation graph in a decentralized manner.The trust assessment among devices can be instantiated by any appropriate assessment technique, for which we focus on remote attestation(integrity verification) in this paper. We prototyped LegIoT for Hyper-ledger Sawtooth and demonstrated through evaluation that the number of trust assessments in the network can be significantly reduced – e.g.,by a factor of 20 for a network of 400 nodes and factor 5 for 1000 nodes.}, author = {Neureither, Jens and Dmitrienko, Alexandra and Koisser, David and Brasser, Ferdinand and Sadeghi, Ahmad-Reza}, booktitle = {European Symposium on Research in Computer Security (ESORICS)}, keywords = {remote-attestation blockchain trust-management sss-group}, title = {LegIoT: Ledgered Trust Management Platform for IoT}, year = 2020 }

Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario. T. Prantl; P. Ten; L. Iffländer; A. Dmitrenko; S. Kounev; C. Krupitzer; in 2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS) (2020).
Acceptance Rate: 27%
[ Abstract ] [ BibTeX ]
 
New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.
@inproceedings{PrTeIfDmKo2020-MASCOTS-GroupEncryption, abstract = {New emerging technologies, such as autonomous driving, intelligent buildings, and smart cities, are promising to revolutionize user experience and offer new services. The world has to undergo large scale deployment of billions of things — cost-efficient intelligent sensors that will be interconnected into extensive networks and will collect and supply data to intelligent algorithms — to make it happen. To date, however, it is challenging to secure such an infrastructure for many-fold reasons, such as resource constraints of things, large scale deployment, manyto-many communication patterns, and dynamically changing communication groups. All these factors rule out most of the state-of-the-art encryption and key-management techniques. Group encryption algorithms are well-suitable for many-tomany communication patterns typical for IoT networks, and many of them can deal with dynamic groups. There are, however, very few constructions that could potentially fulfill the computational and storage constraints of IoT devices while providing sufficient scalability for large networks. The promising candidates, such as construction by Nishat et al. [1], were not evaluated using IoT platforms and under constraints typical for IoT networks. In this paper, we aim to fill this gap and present the evaluation of a state-of-the-art group-oriented encryption scheme by Nishat et al. to identify its applicability to IoT systems. In detail, we provide a measurement workflow, a revised version of the approach, and describe a reproducible hardware testbed. Using this evaluation environment, we analyze the performance of the encryption scheme in a typical IoT scenario from a group member perspective. The results show that all calculation times can be assumed to be constant and are always below 2 seconds. The memory requirement for permanent parameters can also be considered to be constant and are below 8.5 kbit in each case. However, the information that has to be stored temporarily for group updates has turned out to be the bottleneck of the scheme, since their memory requirements increase linearly with the group size.}, author = {Prantl, Thomas and Ten, Peter and Iffländer, Lukas and Dmitrenko, Alexandra and Kounev, Samuel and Krupitzer, Christian}, booktitle = {2020 IEEE 28th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)}, keywords = {Internet_of_Things myown Security t_full descartes Group_Encryption sss-group Performance_Evaluation}, month = {November}, note = {Acceptance Rate: 27\%}, series = {MASCOTS '20}, title = {Evaluating the Performance of a State-of-the-Art Group-oriented Encryption Scheme for Dynamic Groups in an IoT Scenario}, year = 2020 }

Mind the GAP: Security & Privacy Risks of Contact Tracing Apps. L. Baumgärtner; A. Dmitrienko; B. Freisleben; A. Gruler; J. Höchst; J. Kühlberg; M. Mezini; M. Miettinen; A. Muhamedagic; T. D. Nguyen; A. Penning; D. F. Pustelnik; F. Roos; A.-R. Sadeghi; M. Schwarz; C. Uhl; in ArXiv (2020).
[ Abstract ] [ BibTeX ] [ URL ]
 
Contact tracing apps running on mobile devices promise to reduce the manual effort required for identifying infection chains and to increase the tracing accuracy in the presence of COVID-19. Several contract tracing apps have been proposed or deployed in practice. Also Google and Apple have announced their joint effort of providing an API for exposure notification in order to implement decentralized contract tracing apps using Bluetooth Low Energy, the so-called "Google/Apple Proposal", which we abbreviate by "GAP". Some countries have already decided or are planning to base their contact tracing apps on GAP. Several researchers have pointed out potential privacy and security risks related to most of the contact tracing approaches proposed until now, including those that claim privacy protection and are based on GAP. This report makes a first attempt towards providing empirical evidence in real-world scenarios for two such risks discussed in the literature: one concerning privacy, and the other one concerning security. In particular, we focus on a practical analysis of GAP, given that it is the foundation of several tracing apps. We demonstrate that in real-world scenarios the current GAP design is vulnerable to (i) profiling and possibly de-anonymizing infected persons, and (ii) relay-based wormhole attacks that principally can generate fake contacts with the potential of significantly affecting the accuracy of an app-based contact tracing system. For both types of attack, we have built tools that can be easily used on mobile phones or Raspberry Pis (e.g., Bluetooth sniffers). We hope that our findings provide valuable input in the process of testing and certifying contact tracing apps, e.g., as planned for the German Corona-Warn-App, ultimately guiding improvements for secure and privacy-preserving design and implementation of digital contact tracing systems.
@article{mindthegap, abstract = {Contact tracing apps running on mobile devices promise to reduce the manual effort required for identifying infection chains and to increase the tracing accuracy in the presence of COVID-19. Several contract tracing apps have been proposed or deployed in practice. Also Google and Apple have announced their joint effort of providing an API for exposure notification in order to implement decentralized contract tracing apps using Bluetooth Low Energy, the so-called "Google/Apple Proposal", which we abbreviate by "GAP". Some countries have already decided or are planning to base their contact tracing apps on GAP. Several researchers have pointed out potential privacy and security risks related to most of the contact tracing approaches proposed until now, including those that claim privacy protection and are based on GAP. This report makes a first attempt towards providing empirical evidence in real-world scenarios for two such risks discussed in the literature: one concerning privacy, and the other one concerning security. In particular, we focus on a practical analysis of GAP, given that it is the foundation of several tracing apps. We demonstrate that in real-world scenarios the current GAP design is vulnerable to (i) profiling and possibly de-anonymizing infected persons, and (ii) relay-based wormhole attacks that principally can generate fake contacts with the potential of significantly affecting the accuracy of an app-based contact tracing system. For both types of attack, we have built tools that can be easily used on mobile phones or Raspberry Pis (e.g., Bluetooth sniffers). We hope that our findings provide valuable input in the process of testing and certifying contact tracing apps, e.g., as planned for the German Corona-Warn-App, ultimately guiding improvements for secure and privacy-preserving design and implementation of digital contact tracing systems.}, author = {Baumgärtner, Lars and Dmitrienko, Alexandra and Freisleben, Bernd and Gruler, Alexander and Höchst, Jonas and Kühlberg, Joshua and Mezini, Mira and Miettinen, Markus and Muhamedagic, Anel and Nguyen, Thien Duc and Penning, Alvar and Pustelnik, Dermot Frederik and Roos, Filipp and Sadeghi, Ahmad-Reza and Schwarz, Michael and Uhl, Christian}, journal = {ArXiv}, keywords = {technical-reports sss-group}, month = {jun}, title = {Mind the GAP: Security & Privacy Risks of Contact Tracing Apps}, year = 2020 }

Mind the GAP: Security & Privacy Risks of Contact Tracing Apps. L. Baumgärtner; A. Dmitrienko; B. Freisleben; J. Höchst; M. Mezini; M. Miettinen; T. D. Nguyen; A. Penning; F. Roos; A.-R. Sadeghi; M. Schwarz; C. Uhl; (2020).
Accepted to TrustCom 2020, Security Track
[ BibTeX ]
 
@inproceedings{baumgartner2020security, author = {Baumgärtner, Lars and Dmitrienko, Alexandra and Freisleben, Bernd and Höchst, Jonas and Mezini, Mira and Miettinen, Markus and Nguyen, Thien Duc and Penning, Alvar and Roos, Filipp and Sadeghi, Ahmad-Reza and Schwarz, Michael and Uhl, Christian}, keywords = {sss-group}, month = {September}, note = {Accepted to TrustCom 2020, Security Track}, title = {Mind the GAP: Security & Privacy Risks of Contact Tracing Apps}, year = 2020 }

Model-based Performance Predictions for SDN-based Networks: A Case Study. S. Herrnleben; P. Rygielski; J. Grohmann; S. Eismann; T. Hossfeld; S. Kounev; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020).
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.
@inproceedings{HeRyGrEiHoKo-MMB2020-Model-based-SDN-Performance, abstract = {Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.}, address = {Cham}, author = {Herrnleben, Stefan and Rygielski, Piotr and Grohmann, Johannes and Eismann, Simon and Hossfeld, Tobias and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {t_full descartes SDN}, month = {March}, publisher = {Springer}, series = {MMB 2020}, title = {{Model-based Performance Predictions for SDN-based Networks: A Case Study}}, year = 2020 }

A Concept for Crowd-sensed Prediction of Mobile Network Connectivity. S. Herrnleben; B. Zeidler; M. Züfle; C. Krupitzer; S. Kounev; in GI/ITG Workshop on Machine Learning in the Context of Communication Networks 2020 (2020).
[ BibTeX ]
 
@inproceedings{HeZeZuKrKo-FGMLN2020-Prediction-of-Mobile-Network-Connectivity, author = {Herrnleben, Stefan and Zeidler, Bernd and Z{ü}fle, Marwin and Krupitzer, Christian and Kounev, Samuel}, booktitle = {GI/ITG Workshop on Machine Learning in the Context of Communication Networks 2020}, keywords = {Internet_of_Things Networking descartes Cloud Prediction Automated_model_learning Application-aware Self-adaptive-systems t_workshop}, month = {February}, title = {{A Concept for Crowd-sensed Prediction of Mobile Network Connectivity}}, year = 2020 }

Learning to Learn in Collective Adaptive Systems: Mining Design Pattern for Data-driven Reasoning. M. D’Angelo; S. Ghahremani; S. Gerasimou; J. Grohmann; I. Nunes; S. Tomforde; E. Pournaras; in 2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) (2020). 121–126.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Engineering collective adaptive systems (CAS) with learning capabilities is a challenging task due to their multidimensional and complex design space. Data-driven approaches for CAS design could introduce new insights enabling system engineers to manage the CAS complexity more cost-effectively at the design-phase. This paper introduces a systematic approach to reason about design choices and patterns of learning-based CAS. Using data from a systematic literature review, reasoning is performed with a novel application of data-driven methodologies such as clustering, multiple correspondence analysis and decision trees. The reasoning based on past experience as well as supporting novel and innovative design choices are demonstrated.
@conference{DaGhGeGrNuToPo2020-eCas-LearningInCAS, abstract = {Engineering collective adaptive systems (CAS) with learning capabilities is a challenging task due to their multidimensional and complex design space. Data-driven approaches for CAS design could introduce new insights enabling system engineers to manage the CAS complexity more cost-effectively at the design-phase. This paper introduces a systematic approach to reason about design choices and patterns of learning-based CAS. Using data from a systematic literature review, reasoning is performed with a novel application of data-driven methodologies such as clustering, multiple correspondence analysis and decision trees. The reasoning based on past experience as well as supporting novel and innovative design choices are demonstrated.}, author = {D'Angelo, Mirko and Ghahremani, Sona and Gerasimou, Simos and Grohmann, Johannes and Nunes, Ingrid and Tomforde, Sven and Pournaras, Evangelos}, booktitle = {2020 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C)}, keywords = {Statistical_estimation_and_machine_learning myown descartes t_workshop}, pages = {121–126}, publisher = {IEEE}, title = {Learning to Learn in Collective Adaptive Systems: Mining Design Pattern for Data-driven Reasoning}, year = 2020 }

Aggregatable Remote Attestation for IoT. V. Alistarov; Thesis; University of Würzburg. (2020, December).
[ Abstract ] [ BibTeX ]
 
With the recent spike of Internet of Things (IoT) and "smart" devices, there has also been an increase in the amount of attacks on IoT networks. Wide-reaching attacks such as the one from the Mirai botnet in 2016 show how crucial it is to know that a device can be trusted before initiating communication. Remote Attestation (RA) is a proven method for asserting that a device is in a benign state. It is a challenge-response process between two parties, where the first checks the trustworthiness of the second. However, it is characterizable with low scalability a critical issue in the IoT sector. In our work, we model a new RA protocol, called Aggregatable Remote Attestation, which would allow a device to process multiple RA challenges simultaneously. We base it on the already existing SIMPLE architecture and implement it as a Proof-of-Concept (PoC) by modifying the code of the Security MicroVisor the core component of SIMPLE. We evaluate our work in terms of security and performance and show that it greatly outperforms the underlying SIMPLE. We discuss the relevance of our design in relation to the IoT sphere and denote a small set of potential topics for future work and research.
@mastersthesis{alistarov2020aggregatable, abstract = {With the recent spike of Internet of Things (IoT) and "smart" devices, there has also been an increase in the amount of attacks on IoT networks. Wide-reaching attacks such as the one from the Mirai botnet in 2016 show how crucial it is to know that a device can be trusted before initiating communication. Remote Attestation (RA) is a proven method for asserting that a device is in a benign state. It is a challenge-response process between two parties, where the first checks the trustworthiness of the second. However, it is characterizable with low scalability a critical issue in the IoT sector. In our work, we model a new RA protocol, called Aggregatable Remote Attestation, which would allow a device to process multiple RA challenges simultaneously. We base it on the already existing SIMPLE architecture and implement it as a Proof-of-Concept (PoC) by modifying the code of the Security MicroVisor the core component of SIMPLE. We evaluate our work in terms of security and performance and show that it greatly outperforms the underlying SIMPLE. We discuss the relevance of our design in relation to the IoT sphere and denote a small set of potential topics for future work and research.}, author = {Alistarov, Vasil}, keywords = {remote-attestation thesis_supervised_by_sss_member thesis_supervised_by_SSS_member sss-group security}, month = {dec}, school = {University of W{ü}rzburg}, title = {Aggregatable Remote Attestation for IoT}, type = {Bachelor Thesis}, year = 2020 }

The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study. C. Zinner; M. Matzka; R. Leppich; S. Kounev; H. Holmberg; B. Sperlich; in Frontiers in Sports and Active Living (2020). 2 127.
[ BibTeX ]
 
@article{zinner2020impact, author = {Zinner, Christoph and Matzka, Manuel and Leppich, Robert and Kounev, Samuel and Holmberg, HC and Sperlich, Billy}, journal = {Frontiers in Sports and Active Living}, keywords = {myown descartes t_interdisciplinary t_journalmagazine health covid19}, pages = 127, publisher = {Frontiers}, title = {The impact of the German strategy for containment of Coronavirus SARS-CoV-2 on the training characteristics, physical activity, sleep of highly trained kayakers and canoeists: A retrospective observational study}, volume = 2, year = 2020 }

Beyond Microbenchmarks: The SPEC-RG Vision for A Comprehensive Serverless Benchmark. E. van Eyk; J. Scheuner; S. Eismann; C. Abad; A. Iosup; in Companion of the 2020 ACM/SPEC International Conference on Performance Engineering (2020).
[ Abstract ] [ BibTeX ] [ Download ]
 
Serverless computing services, such as Function-as-a-Service (FaaS), hold the attractive promise of a high level of abstraction and high performance, combined with the minimization of operational logic. Several large ecosystems of serverless platforms, both open- and closed-source, aim to realize this promise. Consequently, a lucrative market has emerged. However, the performance trade-offs of these systems are not well-understood. Moreover, it is exactly the high level of abstraction and the opaqueness of the operational-side that make performance evaluation studies of serverless platforms challenging. Learning from the history of IT platforms, we argue that a benchmark for serverless platforms could help address this challenge. We envision a comprehensive serverless benchmark, which we contrast to the narrow focus of prior work in this area. We argue that a comprehensive benchmark will need to take into account more than just runtime overhead, and include notions of cost, realistic workloads, more (open-source) platforms, and cloud integrations. Finally, we show through preliminary real-world experiments how such a benchmark can help compare the performance overhead when running a serverless workload on state-of-the-art platforms.
@inproceedings{10.1145/3185768.3186293, abstract = {Serverless computing services, such as Function-as-a-Service (FaaS), hold the attractive promise of a high level of abstraction and high performance, combined with the minimization of operational logic. Several large ecosystems of serverless platforms, both open- and closed-source, aim to realize this promise. Consequently, a lucrative market has emerged. However, the performance trade-offs of these systems are not well-understood. Moreover, it is exactly the high level of abstraction and the opaqueness of the operational-side that make performance evaluation studies of serverless platforms challenging. Learning from the history of IT platforms, we argue that a benchmark for serverless platforms could help address this challenge. We envision a comprehensive serverless benchmark, which we contrast to the narrow focus of prior work in this area. We argue that a comprehensive benchmark will need to take into account more than just runtime overhead, and include notions of cost, realistic workloads, more (open-source) platforms, and cloud integrations. Finally, we show through preliminary real-world experiments how such a benchmark can help compare the performance overhead when running a serverless workload on state-of-the-art platforms.}, author = {van Eyk, Erwin and Scheuner, Joel and Eismann, Simon and Abad, Cristina and Iosup, Alexandru}, booktitle = {Companion of the 2020 ACM/SPEC International Conference on Performance Engineering}, keywords = {t_visionposition Metrics_and_benchmarking_methodologies Performance descartes SPEC}, title = {{Beyond Microbenchmarks: The SPEC-RG Vision for A Comprehensive Serverless Benchmark}}, year = 2020 }

An IoT Network Emulator for Analyzing the Influence of Varying Network Quality. S. Herrnleben; R. Ailabouni; J. Grohmann; T. Prantl; C. Krupitzer; S. Kounev; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) (2020).
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.
@inproceedings{HeAiGrPrKrKo2020-SIMUtools-IoT-Network-Emulator, abstract = {IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.}, author = {Herrnleben, Stefan and Ailabouni, Rudy and Grohmann, Johannes and Prantl, Thomas and Krupitzer, Christian and Kounev, Samuel}, booktitle = {Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools)}, keywords = {myown t_full descartes internet_of_things networking}, month = {August}, series = {SIMUtools 2020}, title = {An IoT Network Emulator for Analyzing the Influence of Varying Network Quality}, year = 2020 }

Contact Tracing by Giant Data Collectors: Opening Pandora’s Box of Threats to Privacy, Sovereignty and National Security A. Boutet; C. Castelluccia; M. Cunche; A. Dmitrienko; V. Iovino; M. Miettinen; T. D. Nguyen; V. Roca; A.-R. Sadeghi; S. Vaudenay; I. Visconti; M. Vuagnoux; (2020).
[ Abstract ] [ BibTeX ] [ URL ]
 
Many countries have introduced digital contact tracing apps to fight the COVID-19 pandemic. Such apps help to identify contacts between potentially infectious persons automatically and thus bear the promise of reducing the burden on manual contact tracers and increase tracing accuracy in situations in which people have difficulties identifying with whom they have been in contact. A number of different proposals for digital contact tracing systems have been made or deployed, ranging from heavily centralized to completely decentralized approaches, each with its own advantages and disadvantages in terms of tracing effectiveness and impact on user privacy. During the phase of highly dynamic evolution of these approaches, surprisingly, Google and Apple established an unprecedented friendship and agreed on a very special scheme for contact tracing, realizing this in the form of an API called GAEN that they quickly integrated into their mobile operating systems. A multitude of nationally rolled out tracing apps are now based on the GAEN approach. In this paper, we revisit such apps and the GAEN API on which they are built. In particular, we point out a number of very problematic aspects and threats that the GAEN approach creates through its security and privacy weaknesses but also through the threats that it poses on technological sovereignty and the public health system.
@techreport{boutet2020contact, abstract = {Many countries have introduced digital contact tracing apps to fight the COVID-19 pandemic. Such apps help to identify contacts between potentially infectious persons automatically and thus bear the promise of reducing the burden on manual contact tracers and increase tracing accuracy in situations in which people have difficulties identifying with whom they have been in contact. A number of different proposals for digital contact tracing systems have been made or deployed, ranging from heavily centralized to completely decentralized approaches, each with its own advantages and disadvantages in terms of tracing effectiveness and impact on user privacy. During the phase of highly dynamic evolution of these approaches, surprisingly, Google and Apple established an unprecedented friendship and agreed on a very special scheme for contact tracing, realizing this in the form of an API called GAEN that they quickly integrated into their mobile operating systems. A multitude of nationally rolled out tracing apps are now based on the GAEN approach. In this paper, we revisit such apps and the GAEN API on which they are built. In particular, we point out a number of very problematic aspects and threats that the GAEN approach creates through its security and privacy weaknesses but also through the threats that it poses on technological sovereignty and the public health system.}, author = {Boutet, Antoine and Castelluccia, Claude and Cunche, Mathieu and Dmitrienko, Alexandra and Iovino, Vincenzo and Miettinen, Markus and Nguyen, Thien Duc and Roca, Vincent and Sadeghi, Ahmad-Reza and Vaudenay, Serge and Visconti, Ivan and Vuagnoux, Martin}, institution = {{EPFL, Switzerland ; Inria, France ; JMU W{ü}rzburg, Germany ; University of Salerno, Italy ; base23, Geneva, Switzerland ; Technical University of Darmstadt, Germany}}, keywords = {tracing data technical-reports international-conference-workshop-papers-book-chapters sss-group contracts collector}, month = {dec}, title = {{Contact Tracing by Giant Data Collectors: Opening Pandora's Box of Threats to Privacy, Sovereignty and National Security}}, type = {University works}, year = 2020 }

SPEC Research Group Newsletter, vol. 3 no. 1. S. Kounev; A. van Hoorn; A. Bauer; N. Herbst; (2020, April).
Published by Standard Performance Evaluation Corporation (SPEC)
[ BibTeX ] [ URL ] [ Download ]
 
@misc{kounev2020research, author = {Kounev, Samuel and van Hoorn, Andre and Bauer, Andr{é} and Herbst, Nikolas}, keywords = {t_other descartes se2}, month = {April}, note = {Published by Standard Performance Evaluation Corporation (SPEC)}, title = {{SPEC Research Group Newsletter, vol. 3 no. 1}}, year = 2020 }

A Review of Serverless Use Cases and their Characteristics S. Eismann; J. Scheuner; E. van Eyk; M. Schwinger; J. Grohmann; N. Herbst; C. Abad; A. Iosup; (2020).
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
The serverless computing paradigm promises many desirable properties for cloud applications - low-cost, fine-grained deployment, and management-free operation. Consequently, the paradigm has underwent rapid growth: there currently exist tens of serverless platforms and all global cloud providers host serverless operations. To help tune existing platforms, guide the design of new serverless approaches, and overall contribute to understanding this paradigm, in this work we present a long-term, comprehensive effort to identify, collect, and characterize serverless use cases. We survey 89 use cases, sourced from white and grey literature, and from consultations with experts in areas such as scientific computing. We study each use case using 24 characteristics, including general aspects, but also workload, application, and requirements. When the use cases employ workflows, we further analyze their characteristics. Overall, we hope our study will be useful for both academia and industry, and encourage the community to further share and communicate their use cases.
@techreport{eismann2020review, abstract = {The serverless computing paradigm promises many desirable properties for cloud applications - low-cost, fine-grained deployment, and management-free operation. Consequently, the paradigm has underwent rapid growth: there currently exist tens of serverless platforms and all global cloud providers host serverless operations. To help tune existing platforms, guide the design of new serverless approaches, and overall contribute to understanding this paradigm, in this work we present a long-term, comprehensive effort to identify, collect, and characterize serverless use cases. We survey 89 use cases, sourced from white and grey literature, and from consultations with experts in areas such as scientific computing. We study each use case using 24 characteristics, including general aspects, but also workload, application, and requirements. When the use cases employ workflows, we further analyze their characteristics. Overall, we hope our study will be useful for both academia and industry, and encourage the community to further share and communicate their use cases.}, author = {Eismann, Simon and Scheuner, Joel and van Eyk, Erwin and Schwinger, Maximilian and Grohmann, Johannes and Herbst, Nikolas and Abad, Cristina and Iosup, Alexandru}, keywords = {myown descartes t_techreport}, month = {June}, publisher = {SPEC RG}, title = {A Review of Serverless Use Cases and their Characteristics}, year = 2020 }

Toward a Framework for Self-Learning Adaptation Planning through Optimization. V. Lesch; in Organic Computing: Doctoral Dissertation Colloquium 2020 (2020).
[ BibTeX ]
 
@incollection{lesch2020toward, author = {Lesch, Veronika}, booktitle = {Organic Computing: Doctoral Dissertation Colloquium 2020}, keywords = {myown industry_4_0 self-aware-computing optimization t_full descartes intelligent_transportation_systems cyber-physical_systems}, title = {Toward a Framework for Self-Learning Adaptation Planning through Optimization}, year = 2020 }

Introduction to the Special Issue ``Applications in Self-Aware Computing Systems and their Evaluation’’. C. Krupitzer; B. Eberhardinger; I. Gerostathopoulos; C. Raibulet; (2020).
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
The joint 1st Workshop on Evaluations and Measurements in Self-Aware Computing Systems (EMSAC 2019) and Workshop on Self-Aware Computing (SeAC) was held as part of the FAS* conference alliance in conjunction with the 16th IEEE International Conference on Autonomic Computing (ICAC) and the 13th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO) in Umea, Sweden on 20 June 2019. The goal of this one-day workshop was to bring together researchers and practitioners from academic environments and from the industry to share their solutions, ideas, visions, and doubts in self-aware computing systems in general and in the evaluation and measurements of such systems in particular. The workshop aimed to enable discussions, partnerships, and collaborations among the participants. This special issue follows the theme of the workshop. It contains extended versions of workshop presentations as well as additional contributions.
@misc{KrEbGeRa2020_Computers_seacSI, abstract = {The joint 1st Workshop on Evaluations and Measurements in Self-Aware Computing Systems (EMSAC 2019) and Workshop on Self-Aware Computing (SeAC) was held as part of the FAS* conference alliance in conjunction with the 16th IEEE International Conference on Autonomic Computing (ICAC) and the 13th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO) in Umea, Sweden on 20 June 2019. The goal of this one-day workshop was to bring together researchers and practitioners from academic environments and from the industry to share their solutions, ideas, visions, and doubts in self-aware computing systems in general and in the evaluation and measurements of such systems in particular. The workshop aimed to enable discussions, partnerships, and collaborations among the participants. This special issue follows the theme of the workshop. It contains extended versions of workshop presentations as well as additional contributions.}, author = {Krupitzer, Christian and Eberhardinger, Benedikt and Gerostathopoulos, Ilias and Raibulet, Claudia}, journal = {MDPI Computers}, keywords = {descartes se2}, number = 1, publisher = {MDPI}, title = {{Introduction to the Special Issue ``Applications in Self-Aware Computing Systems and their Evaluation''}}, volume = 9, year = 2020 }

Microservices: A Performance Tester’s Dream or Nightmare?. S. Eismann; C.-P. Bezemer; W. Shang; D. Okanovic; A. van Hoorn; in Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE) (2020).
Acceptance Rate: 23.4% (15/64), ACM Artifacts Evaluated — Functional
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
In recent years, there has been a shift in software development towards microservice-based architectures, which consist of small services that focus on one particular functionality. Many companies are migrating their applications to such architectures to reap the benefits of microservices, such as increased flexibility, scalability and a smaller granularity of the offered functionality by a service. On the one hand, the benefits of microservices for functional testing are often praised, as the focus on one functionality and their smaller granularity allow for more targeted and more convenient testing. On the other hand, using microservices has their consequences (both positive and negative) on other types of testing, such as performance testing. Performance testing is traditionally done by establishing the baseline performance of a software version, which is then used to compare the performance testing results of later software versions. However, as we show in this paper, establishing such a baseline performance is challenging in microservice applications. In this paper, we discuss the benefits and challenges of microservices from a performance testers point of view. Through a series of experiments on the TeaStore application, we demonstrate how microservices affect the performance testing process, and we demonstrate that it is not straightforward to achieve reliable performance testing results for a microservice application.
@inproceedings{EiBeShOkHo2020-ICPE-ServerlessWorkflows, abstract = {In recent years, there has been a shift in software development towards microservice-based architectures, which consist of small services that focus on one particular functionality. Many companies are migrating their applications to such architectures to reap the benefits of microservices, such as increased flexibility, scalability and a smaller granularity of the offered functionality by a service. On the one hand, the benefits of microservices for functional testing are often praised, as the focus on one functionality and their smaller granularity allow for more targeted and more convenient testing. On the other hand, using microservices has their consequences (both positive and negative) on other types of testing, such as performance testing. Performance testing is traditionally done by establishing the baseline performance of a software version, which is then used to compare the performance testing results of later software versions. However, as we show in this paper, establishing such a baseline performance is challenging in microservice applications. In this paper, we discuss the benefits and challenges of microservices from a performance testers point of view. Through a series of experiments on the TeaStore application, we demonstrate how microservices affect the performance testing process, and we demonstrate that it is not straightforward to achieve reliable performance testing results for a microservice application.}, author = {Eismann, Simon and Bezemer, Cor-Paul and Shang, Weiyi and Okanovic, Dusan and van Hoorn, Andre}, booktitle = {Proceedings of the 2020 ACM/SPEC International Conference on Performance Engineering (ICPE)}, keywords = {t_full descartes se2}, month = {April}, note = {{Acceptance Rate: 23.4% (15/64), ACM Artifacts Evaluated — Functional}}, series = {ICPE'20}, title = {{Microservices: A Performance Tester's Dream or Nightmare?}}, year = 2020 }

A Taxonomy of Techniques for SLO Failure Prediction in Software Systems. J. Grohmann; N. Herbst; A. Chalbani; Y. Arian; N. Peretz; S. Kounev; in Computers (2020). 9(1) 10.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ]
 
Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.
@article{grohmann2020-MDPI-taxonomy, abstract = {Failure prediction is an important aspect of self-aware computing systems. Therefore, a multitude of different approaches has been proposed in the literature over the past few years. In this work, we propose a taxonomy for organizing works focusing on the prediction of Service Level Objective (SLO) failures. Our taxonomy classifies related work along the dimensions of the prediction target (e.g., anomaly detection, performance prediction, or failure prediction), the time horizon (e.g., detection or prediction, online or offline application), and the applied modeling type (e.g., time series forecasting, machine learning, or queueing theory). The classification is derived based on a systematic mapping of relevant papers in the area. Additionally, we give an overview of different techniques in each sub-group and address remaining challenges in order to guide future research.}, author = {Grohmann, Johannes and Herbst, Nikolas and Chalbani, Avi and Arian, Yair and Peretz, Noam and Kounev, Samuel}, journal = {Computers}, keywords = {descartes performance t_journalmagazine se2 automated_model_learning}, number = 1, pages = 10, publisher = {Multidisciplinary Digital Publishing Institute (MDPI)}, title = {A Taxonomy of Techniques for SLO Failure Prediction in Software Systems}, volume = 9, year = 2020 }

An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series. A. Bauer; M. Züfle; J. Grohmann; N. Schmitt; N. Herbst; S. Kounev; in Proceedings of the ACM/SPEC International Conference on Performance Engineering (2020). 48–55.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.
@inproceedings{BaZuGrScHeKo-ICPE20-Seasonal-Forecast, abstract = {Due to the fast-paced and changing demands of their users, computing systems require autonomic resource management. To enable proactive and accurate decision-making for changes causing a particular overhead, reliable forecasts are needed. In fact, choosing the best performing forecasting method for a given time series scenario is a crucial task. Taking the "No-Free-Lunch Theorem" into account, there exists no forecasting method that performs best on all types of time series. To this end, we propose an automated approach that (i) extracts characteristics from a given time series, (ii) selects the best-suited machine learning method based on recommendation, and finally, (iii) performs the forecast. Our approach offers the benefit of not relying on a single method with its possibly inaccurate forecasts. In an extensive evaluation, our approach achieves the best forecasting accuracy.}, address = {New York, NY, USA}, author = {Bauer, Andr{é} and Z{ü}fle, Marwin and Grohmann, Johannes and Schmitt, Norbert and Herbst, Nikolas and Kounev, Samuel}, booktitle = {Proceedings of the ACM/SPEC International Conference on Performance Engineering}, keywords = {Online_monitoring_and_forecasting t_short descartes Prediction PRISMA}, month = {April}, pages = {48–55}, publisher = {Association for Computing Machinery (ACM)}, series = {ICPE '20}, title = {{An Automated Forecasting Framework based on Method Recommendation for Seasonal Time Series}}, year = 2020 }

Methodological Principles for Reproducible Performance Evaluation in Cloud Computing. A. V. Papadopoulos; L. Versluis; A. Bauer; N. Herbst; J. von Kistowski; A. Ali-Eldin; C. L. Abad; J. N. Amaral; P. Tuma; A. Iosup; in Software Engineering 2020, Fachtagung des GI-Fachbereichs Softwaretechnik, 24.-28. Februar 2020, Innsbruck, Austria, M. Felderer, W. Hasselbring, R. Rabiser, R. Jung (eds.) (2020). (Vol. P-300) 93–94.
[ BibTeX ] [ URL ] [ Download ]
 
@inproceedings{DBLP:conf/se/PapadopoulosVBH20, author = {Papadopoulos, Alessandro Vittorio and Versluis, Laurens and Bauer, Andr{{é}} and Herbst, Nikolas and von Kistowski, J{{ó}}akim and Ali{-}Eldin, Ahmed and Abad, Cristina L. and Amaral, Jos{{é}} Nelson and Tuma, Petr and Iosup, Alexandru}, booktitle = {Software Engineering 2020, Fachtagung des GI-Fachbereichs Softwaretechnik, 24.-28. Februar 2020, Innsbruck, Austria}, editor = {Felderer, Michael and Hasselbring, Wilhelm and Rabiser, Rick and Jung, Reiner}, keywords = {t_extendedabstract descartes se2}, pages = {93–94}, publisher = {Gesellschaft f{{ü}}r Informatik e.V.}, series = {{LNI}}, title = {{Methodological Principles for Reproducible Performance Evaluation in Cloud Computing}}, volume = {{P-300}}, year = 2020 }

Time Series Forecasting for Self-Aware Systems. A. Bauer; M. Züfle; N. Herbst; A. Zehe; A. Hotho; S. Kounev; in Proceedings of the IEEE (2020). 108(7) 1068–1093.
[ BibTeX ] [ DOI ]
 
@article{BaZuHeZeHoKo-PIEEE-Forecasting, author = {Bauer, Andr{é} and Z{ü}fle, Marwin and Herbst, Nikolas and Zehe, Albin and Hotho, Andreas and Kounev, Samuel}, journal = {Proceedings of the IEEE}, keywords = {Telescope Metrics_and_benchmarking_methodologies Online_monitoring_and_forecasting descartes Self-aware-computing Prediction Elasticity t_journalmagazine}, month = 7, number = 7, pages = {1068 - 1093}, title = {{Time Series Forecasting for Self-Aware Systems}}, volume = 108, year = 2020 }

Quantifying measurement quality and load distribution in Tor. A. Greubel; S. Pohl; S. Kounev; in Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020) (2020).
[ Abstract ] [ BibTeX ] [ Download ]
 
Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.
@article{greubel2020quantifying, abstract = {Tor is a widely used anonymization network. Traffic is routed over different relay nodes to conceal the communication partners. However, if a single relay handles too much traffic, de-anonymization attacks are possible. The Tor Load Balancing Mechanism (TLBM) is responsible for balanced and secure load distribution. It must verify that relays cannot attract more traffic than they should by lying about their self-reported bandwidth. This work shows that the current bandwidth measurement method used for bandwidth verification is not suitable to verify the bandwidth of many relays. Most importantly, multiple measurements of high-bandwidth relays are uncorrelated to each other. Furthermore, we analyze the current load distribution in Tor. We show that the current load distribution reduces the resources necessary for several large-scale de-anonymization attacks by more than 80\%. Additionally, as Tor favors fast relays during path selection, verifiable relays only handle a small fraction of Tor’s traffic. More precisely, we show that only 7.21\% of all paths consist of entry and exit relays verifiable by measurements. We discuss these results’ security implications and argue that future TLBM research should focus at least as much on secure load distribution as on high traffic performance.}, author = {Greubel, Andre and Pohl, Steffen and Kounev, Samuel}, journal = {Proceedings of the 36th Annual Computer Security Applications Conference (ACSAC 2020)}, keywords = {balancing t_full TLBM descartes tor}, title = {Quantifying measurement quality and load distribution in Tor}, year = 2020 }

Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite. N. Schmitt; J. Bucek; K.-D. Lange; S. Kounev; in Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020) (2020).
[ BibTeX ] [ URL ] [ Download ]
 
@inproceedings{ScLaBuKo-ICPE20-SPEC-CPU2017, address = {New York, NY, USA}, author = {Schmitt, Norbert and Bucek, James and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 11th ACM/SPEC International Conference on Performance Engineering (ICPE 2020)}, keywords = {myown Performance descartes SPEC t_wip Power-energy_efficient_computing Power}, month = {April}, publisher = {ACM}, title = {{Energy Efficiency Analysis of Compiler Optimizations on the SPEC CPU 2017 Benchmark Suite}}, year = 2020 }

A Survey on Predictive Maintenance for Industry 4.0 C. Krupitzer; T. Wagenhals; M. Züfle; V. Lesch; D. Schäfer; A. Mozaffarin; J. Edinger; C. Becker; S. Kounev; (2020).
[ BibTeX ] [ URL ] [ Download ]
 
@techreport{Krupitzer2020ASurveyPdM, author = {Krupitzer, Christian and Wagenhals, Tim and Z{ü}fle, Marwin and Lesch, Veronika and Sch{ä}fer, Dominik and Mozaffarin, Amin and Edinger, Janick and Becker, Christian and Kounev, Samuel}, institution = {Universit{ä}t W{ü}rzburg and University of Mannheim and Syntax Systems GmbH and MOZYS Engineering GmbH}, journal = {arXiv}, keywords = {Internet_of_Things Survey Cyber-Physical_Systems Online_monitoring_and_forecasting descartes Prediction Industry_4_0 t_techreport t_interdisciplinary Reliability}, month = {feb}, title = {{A Survey on Predictive Maintenance for Industry 4.0}}, year = 2020 }

To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows. M. Züfle; C. Krupitzer; F. Erhard; J. Grohmann; S. Kounev; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020). 19–36.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.
@inproceedings{ZuKrErGrKo-MMB2020-ToFailOrNotToFail, abstract = {Due to the increasing size of today's data centers as well as the expectation of 24/7 availability, the complexity in the administration of hardware continuously icreases. Techniques as the Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) support the monitoring of the hardware. However, those techniques often lack algorithms for intelligent data analytics. Especially, the integration of machine learning to identify potential failures in advance seems to be promising to reduce administration overhead. In this work, we present three machine learning approaches to (i) identify imminent failures, (ii) predict time windows for failures, as well as (iii) predict the exact time-to-failure. In a case study with real data from 369 hard disks, we achieve an F1-score of up to 98.0% and 97.6% for predicting potential failures with two or multiple time windows, respectively, and a hit rate of 84.9% (with a mean absolute error of 4.5 hours) for predicting the time-to-failure.}, address = {Cham}, author = {Z{ü}fle, Marwin and Krupitzer, Christian and Erhard, Florian and Grohmann, Johannes and Kounev, Samuel}, booktitle = {Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems}, keywords = {Statistical_estimation_and_machine_learning Internet_of_Things Cyber-Physical_Systems Performance t_full descartes Prediction PRISMA Reliability}, month = {March}, pages = {19–36}, publisher = {Springer}, series = {MMB 2020}, title = {{To Fail Or Not To Fail: Predicting Hard Disk Drive Failure Time Windows}}, year = 2020 }

Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field. A. Bauer; M. Züfle; N. Herbst; S. Kounev; V. Curtef; in Proceedings of the 36th International Conference on Data Engineering (ICDE) (2020).
[ BibTeX ] [ Download ]
 
@inproceedings{BaZuHeKoCu-ICDE-Telescope, author = {Bauer, Andr{é} and Z{ü}fle, Marwin and Herbst, Nikolas and Kounev, Samuel and Curtef, Valentin}, booktitle = {Proceedings of the 36th International Conference on Data Engineering (ICDE)}, keywords = {Tool Telescope Online_monitoring_and_forecasting t_short descartes Prediction}, month = {April}, title = {Telescope: An Automatic Feature Extraction and Transformation Approach for Time Series Forecasting on a Level-Playing Field}, year = 2020 }

A Survey on Human Machine Interaction in Industry 4.0 C. Krupitzer; S. Müller; V. Lesch; M. Züfle; J. Edinger; A. Lemken; D. Schäfer; S. Kounev; C. Becker; (2020).
[ BibTeX ] [ URL ] [ Download ]
 
@techreport{Krupitzer2020ASurvey, author = {Krupitzer, Christian and M{ü}ller, Sebastian and Lesch, Veronika and Z{ü}fle, Marwin and Edinger, Janick and Lemken, Alexander and Sch{ä}fer, Dominik and Kounev, Samuel and Becker, Christian}, institution = {Universit{ä}t W{ü}rzburg and University of Mannheim and ioxp GmbH and Syntax Systems GmbH}, journal = {arXiv}, keywords = {Survey descartes Industry_4_0 t_techreport t_interdisciplinary}, month = {feb}, title = {{A Survey on Human Machine Interaction in Industry 4.0}}, year = 2020 }

Towards Self-Aware Multirotor Formations. D. Kaiser; V. Lesch; J. Rothe; M. Strohmeier; F. Spiess; C. Krupitzer; S. Montenegro; S. Kounev; in Computers (2020). 9(7)
Special Issue on Self-Aware Computing
[ BibTeX ] [ Download ]
 
@article{Kaiser2020Towards, author = {Kaiser, Dennis and Lesch, Veronika and Rothe, Julian and Strohmeier, Michael and Spiess, Florian and Krupitzer, Christian and Montenegro, Sergio and Kounev, Samuel}, journal = {Computers}, keywords = {Internet_of_Things Cyber-Physical_Systems Design_of_software_and_systems Analytical_and_simulation-based_analysis Simulation descartes Self-aware-computing t_interdisciplinary t_journalmagazine Intelligent_Transportation_Systems}, month = {Februar}, note = {Special Issue on Self-Aware Computing}, number = 7, publisher = {MDPI}, title = {{Towards Self-Aware Multirotor Formations}}, volume = 9, year = 2020 }

Intra-individual physiological response of recreational runners to different training mesocycles: a randomized cross-over study. P. Düking; H.-C. Holmberg; P. Kunz; R. Leppich; B. Sperlich; in European Journal of Applied Physiology (2020). 1–9.
[ BibTeX ]
 
@article{duking2020intra, author = {D{ü}king, Peter and Holmberg, Hans-Christer and Kunz, Philipp and Leppich, Robert and Sperlich, Billy}, journal = {European Journal of Applied Physiology}, keywords = {myown descartes sport t_interdisciplinary t_journalmagazine}, pages = {1–9}, publisher = {Springer}, title = {Intra-individual physiological response of recreational runners to different training mesocycles: a randomized cross-over study}, year = 2020 }

2019 [ nach oben ]

Performance Oriented Dynamic Bypassing for Intrusion Detection Systems. L. Iffländer; J. Stoll; N. Rawtani; V. Lesch; K.-D. Lange; S. Kounev; in Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering (2019). 159–166.
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.
@inproceedings{Ifflaender2019_PerformanceOrientedDynamicBypassingforIntrusionDetectionSystems, abstract = {Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Moore's law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.}, address = {New York, NY, USA}, author = {Iffländer, Lukas and Stoll, Jonathan and Rawtani, Nishant and Lesch, Veronika and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Proceedings of the 2019 ACM/SPEC International Conference on Performance Engineering}, keywords = {Metrics_and_benchmarking_methodologies Resource_management Networking Security Performance t_short descartes Self-aware-computing EvIDencE Self-adaptive-systems Optimization}, pages = {159–166}, publisher = {ACM}, series = {ICPE '19}, title = {Performance Oriented Dynamic Bypassing for Intrusion Detection Systems}, year = 2019 }

Improving the Energy Efficiency of IoT-Systems and its Software. N. Schmitt; in Organic Computing: Doctoral Dissertation Colloquium 2018, S. Tomforde, B. Sick (eds.) (2019).
[ BibTeX ] [ Download ]
 
@incollection{Sc2015-EEIoTSoftware, author = {Schmitt, Norbert}, booktitle = {{Organic Computing: Doctoral Dissertation Colloquium 2018}}, editor = {Tomforde, Sven and Sick, Bernhard}, keywords = {myown t_visionposition descartes Self-aware-computing Cloud Power-energy_efficient_computing Power}, month = {April}, publisher = {kassel university press GmbH}, title = {{Improving the Energy Efficiency of IoT-Systems and its Software}}, year = 2019 }

Integrating Statistical Response Time Models in Architectural Performance Models. S. Eismann; J. Grohmann; J. Walter; J. von Kistowski; S. Kounev; in Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA) (2019). 71–80.
Acceptance Rate: 21,9% (21/96)
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.
@inproceedings{EiGrWaKiKo2019-ICSA-Integrating, abstract = {Performance predictions enable software architects to optimize the performance of a software system early in the development cycle. Architectural performance models and statistical response time models are commonly used to derive these performance predictions. However, both methods have significant downsides: Statistical response time models can only predict scenarios for which training data is available, making the prediction of previously unseen system configurations infeasible. In contrast, the time required to simulate an architectural performance model increases exponentially with both system size and level of modeling detail, making the analysis of large, detailed models challenging. Existing approaches use statistical response time models in architectural performance models to avoid modeling subsystems that are difficult or time-consuming to model, yet they do not consider simulation time. In this paper, we propose to model software systems using classical queuing theory and statistical response time models in parallel. This approach allows users to tailor the model for each analysis run, based on the performed adaptations and the requested performance metrics. Our approach enables faster model solution compared to traditional performance models while retaining their ability to predict previously unseen scenarios. In our experiments we observed speedups of up to 94.8%, making the analysis of much larger and more detailed systems feasible.}, author = {Eismann, Simon and Grohmann, Johannes and Walter, J{ü}rgen and von Kistowski, J{ó}akim and Kounev, Samuel}, booktitle = {Proceedings of the 2019 IEEE International Conference on Software Architecture (ICSA)}, keywords = {Statistical_estimation_and_machine_learning DML Performance t_full Analytical_and_simulation-based_analysis descartes Prediction Formal_architecture_modeling Meta-models QPN QPME Simulation PRISMA}, month = {March}, note = {Acceptance Rate: 21,9\% (21/96)}, pages = {71–80}, publisher = {IEEE}, title = {{Integrating Statistical Response Time Models in Architectural Performance Models}}, year = 2019 }

Performance Influence of Security Function Chain Ordering. L. Iffländer; N. Fella; in Companion of the 2019 ACM/SPEC International Conference on Performance Engineering (2019). 45–46.
[ Abstract ] [ BibTeX ] [ URL ] [ Download ]
 
In modern days security systems often reach their performance peak and limit the protected application. Utilizing the available resources for security more efficiently is becoming more critical. In this paper, we introduce the claim, that no static security function chain is optimal in every situation. First experiments prove our claim.
@inproceedings{Ifflaender2019_PerformanceInfluenceofSecurityFunctionChainOrdering, abstract = {In modern days security systems often reach their performance peak and limit the protected application. Utilizing the available resources for security more efficiently is becoming more critical. In this paper, we introduce the claim, that no static security function chain is optimal in every situation. First experiments prove our claim.}, address = {New York, NY, USA}, author = {Iffländer, Lukas and Fella, Nicolas}, booktitle = {Companion of the 2019 ACM/SPEC International Conference on Performance Engineering}, keywords = {Metrics_and_benchmarking_methodologies Resource_management Networking Performance descartes Optimization Security Online_monitoring_and_forecasting Self-aware-computing EvIDencE Application-aware SDN t_poster Reliability Self-adaptive-systems}, pages = {45–46}, publisher = {ACM}, series = {ICPE '19}, title = {Performance Influence of Security Function Chain Ordering}, year = 2019 }

Methoden und Messverfahren für Automatisches Skalieren in Elastischen Cloud Umgebungen. N. Herbst; (2019, June).
[ BibTeX ] [ Download ]
 
@misc{talk-Herbst-GIAwardNomination-Dagstuhl, author = {Herbst, Nikolas}, howpublished = {GI Dissertationspreis: Seminar f{ü}r Nominierte Doktoranden, Dagstuhl, Germany, June 28, 2019}, keywords = {Telescope Metrics_and_benchmarking_methodologies Resource_management descartes Prediction Chameleon Instrumentation_profiling_and_workload_characterization LIMBO Online_monitoring_and_forecasting se2_talk Cloud Award BUNGEE Virtualization t_bookchapter}, month = {June}, title = {{Methoden und Messverfahren f{ü}r Automatisches Skalieren in Elastischen Cloud Umgebungen}}, year = 2019 }

TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research. S. Eismann; (2019, February).
[ Abstract ] [ BibTeX ] [ Download ]
 
Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed micro-service applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Unfortunately, existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. In this talk, we introduce the TeaStore, a state-of-the-art test and reference application with a micro-service architecture. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. Based on three experiments, we demonstrate TeaStore's use in three different research contexts: performance modeling, cloud resource management, and energy efficiency analysis.
@misc{talk-Eismann-SE2019, abstract = {Researchers propose and employ various methods to analyze, model, optimize and manage modern distributed micro-service applications. In order to demonstrate and evaluate these methods in realistic scenarios, researchers rely on reference applications. These applications should offer a range of different behaviors, degrees of freedom allowing for customization and should use a modern and representative technology stack. Unfortunately, existing testing and benchmarking applications are either outdated, designed for specific testing scenarios, or do not offer the necessary degrees of freedom. In this talk, we introduce the TeaStore, a state-of-the-art test and reference application with a micro-service architecture. TeaStore offers services with different performance characteristics and many degrees of freedom regarding deployment and configuration to be used as a benchmarking framework for researchers. Based on three experiments, we demonstrate TeaStore's use in three different research contexts: performance modeling, cloud resource management, and energy efficiency analysis.}, author = {Eismann, Simon}, howpublished = {Multikonferenz Software Engineering & Management 2019}, keywords = {Tool Design_of_software_and_systems se2_talk t_talk SOA descartes Cloud}, month = {February}, title = {{TeaStore - A Micro-Service Application for Benchmarking, Modeling and Resource Management Research}}, year = 2019 }

WheelShare: Crowd-sensed Surface Classification for Accessible Routing. J. Edinger; V. Raychoudhury; A. Hofmann; A. Wachner; C. Becker; C. Krupitzer; in Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) (2019).
[ Abstract ] [ BibTeX ] [ Download ]
 
Accessible path routing for wheeled mobility is an important problem given the permanent and temporary obstacles in the built environment. Existing research works have focused on identifying several obstacles as well as facilities such as crosswalks with traffic signals using smartphone based sensing or crowd-sourcing and used those knowledge to generate accessible routes. While all those research works have identified the nature of the road or sidewalk surface (even/uneven) as a major accessibility concern, there is no in-depth empirical research regarding how the accessibility varies from surface to surface. In this work, we propose a hybrid system, called WheelShare which can clearly classify (in/)accessible surfaces.We have analyzed vibration (using an aide-propelled manual wheelchair) data from 11 different standard indoor and outdoor surfaces found in the built environment and classified them using WheelShare into five different categories across the accessibility spectrum with an accuracy of up to 95\%.Our objective knowledge of surfaces is then applied to generate an accessible route through the best possible surface depending on user and wheelchair requirements. WheelShare is 1) scalable, as it uses crowd-sensing to collect data, 2) dynamic, as the data gets constantly updated, and 3) objective, as it uses an objective and data-centric approach.
@inproceedings{EdRaHoWaBeKr2019-TSUM-wheelShareConcept, abstract = {Accessible path routing for wheeled mobility is an important problem given the permanent and temporary obstacles in the built environment. Existing research works have focused on identifying several obstacles as well as facilities such as crosswalks with traffic signals using smartphone based sensing or crowd-sourcing and used those knowledge to generate accessible routes. While all those research works have identified the nature of the road or sidewalk surface (even/uneven) as a major accessibility concern, there is no in-depth empirical research regarding how the accessibility varies from surface to surface. In this work, we propose a hybrid system, called WheelShare which can clearly classify (in/)accessible surfaces.We have analyzed vibration (using an aide-propelled manual wheelchair) data from 11 different standard indoor and outdoor surfaces found in the built environment and classified them using WheelShare into five different categories across the accessibility spectrum with an accuracy of up to 95\%.Our objective knowledge of surfaces is then applied to generate an accessible route through the best possible surface depending on user and wheelchair requirements. WheelShare is 1) scalable, as it uses crowd-sensing to collect data, 2) dynamic, as the data gets constantly updated, and 3) objective, as it uses an objective and data-centric approach.}, address = {Kyoto, Japan}, author = {Edinger, Janick and Raychoudhury, Vaskar and Hofmann, Alexandra and Wachner, Anton and Becker, Christian and Krupitzer, Christian}, booktitle = {Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops)}, keywords = {Internet_of_Things descartes Self-aware-computing Self-adaptive-systems Optimization}, title = {{WheelShare: Crowd-sensed Surface Classification for Accessible Routing}}, year = 2019 }

On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework. M. D’Angelo; S. Gerasimou; S. Ghahremani; J. Grohmann; I. Nunes; E. Pournaras; S. Tomforde; in Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (2019). 13–24.
[ Abstract ] [ BibTeX ] [ URL ] [ DOI ] [ Download ]
 
Collective self-adaptive systems (CSAS) are distributed and interconnected systems composed of multiple agents that can perform complex tasks such as environmental data collection, search and rescue operations, and discovery of natural resources. By providing individual agents with learning capabilities, CSAS can cope with challenges related to distributed sensing and decision-making and operate in uncertain environments. This unique characteristic of CSAS enables the collective to exhibit robust behaviour while achieving system-wide and agent-specific goals. Although learning has been explored in many CSAS applications, selecting suitable learning models and techniques remains a significant challenge that is heavily influenced by expert knowledge. We address this gap by performing a multifaceted analysis of existing CSAS with learning capabilities reported in the literature. Based on this analysis, we introduce a 3D framework that illustrates the learning aspects of CSAS considering the dimensions of autonomy, knowledge access, and behaviour, and facilitates the selection of learning techniques and models. Finally, using example applications from this analysis, we derive open challenges and highlight the need for research on collaborative, resilient and privacy-aware mechanisms for CSAS.
@inproceedings{SEAMS2019-CollectiveLearningSurvey, abstract = {Collective self-adaptive systems (CSAS) are distributed and interconnected systems composed of multiple agents that can perform complex tasks such as environmental data collection, search and rescue operations, and discovery of natural resources. By providing individual agents with learning capabilities, CSAS can cope with challenges related to distributed sensing and decision-making and operate in uncertain environments. This unique characteristic of CSAS enables the collective to exhibit robust behaviour while achieving system-wide and agent-specific goals. Although learning has been explored in many CSAS applications, selecting suitable learning models and techniques remains a significant challenge that is heavily influenced by expert knowledge. We address this gap by performing a multifaceted analysis of existing CSAS with learning capabilities reported in the literature. Based on this analysis, we introduce a 3D framework that illustrates the learning aspects of CSAS considering the dimensions of autonomy, knowledge access, and behaviour, and facilitates the selection of learning techniques and models. Finally, using example applications from this analysis, we derive open challenges and highlight the need for research on collaborative, resilient and privacy-aware mechanisms for CSAS.}, author = {{D'Angelo}, M. and {Gerasimou}, S. and {Ghahremani}, S. and {Grohmann}, J. and {Nunes}, I. and {Pournaras}, E. and {Tomforde}, S.}, booktitle = {Proceedings of the 14th International Symposium on Software Engineering for Adaptive and Self-Managing Systems}, keywords = {Survey Design_of_software_and_systems t_full descartes Self-aware-computing Automated_model_learning Self-adaptive-systems}, month = {may}, pages = {13–24}, publisher = {IEEE Press}, series = {SEAMS '19}, title = {On Learning in Collective Self-Adaptive Systems: State of Practice and a 3D Framework}, year = 2019 }

A Concept for an Adaptive Communication Middleware for Car-2-Cloud Applications. S. Herrnleben; (2019, April).
[ BibTeX ] [ Download ]
 
@misc{He2019-SelPhyS-Car2Cloud, author = {Herrnleben, Stefan}, howpublished = {Talk at Workshop on Self-Awareness in Cyber-Physical Systems 2019}, keywords = {Internet_of_Things descartes Application-aware Talks Self-adaptive-systems t_workshop}, month = {April}, title = {{A Concept for an Adaptive Communication Middleware for Car-2-Cloud Applications}}, year = 2019 }