Deutsch Intern

Chair of Computer Science II - Software Engineering

Publications

Journal and Magazine Articles

Full Papers

Quantifying Data Leakage in Failure Prediction Tasks. Grillmeyer, Daniel; Hadry, Marius; Lesch, Veronika; Borst, Vanessa; Leppich, Robert; Bauer, Andr{é}; Kounev, Samuel; in Proceedings of the 16th ACM/SPEC International Conference on Performance Engineering (2025). 334–346. Association for Computing Machinery, Toronto ON, Canada.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
With the ever increasing importance of cloud computing and a strong focus on reliable data centers, a high amount of research has been done on failure prediction for hard disk drives. The collection of monitoring data, such as SMART statistics (Self-Monitoring, Analysis, and Reporting Technology) from operational HDDs, enables operators to obtain predictions about the expected remaining useful life. Numerous methods for HDD failure prediction have been published in recent years, and their evaluation has shown decent results. However, a naive splitting into training and test sets can lead to data leakage and, thus, over-optimistic results that cannot be achieved on the data of scientific interest. In this paper, we propose a novel data leakage measure for quantifying the amount of data leakage in training and test datasets. Further, we define four splitting techniques and evaluate our measure in terms of the performance optimism of classification models with respect to these different splitting strategies. Our results consistently show that splitting techniques prone to data leakage induce an overestimation of predictive performance. Overall, we were able to show the usefulness of the defined data leakage measure, as well as its connection with different splitting techniques and the performance optimism of prediction models.

@inproceedings{10.1145/3676151.3719368, abstract = {With the ever increasing importance of cloud computing and a strong focus on reliable data centers, a high amount of research has been done on failure prediction for hard disk drives. The collection of monitoring data, such as SMART statistics (Self-Monitoring, Analysis, and Reporting Technology) from operational HDDs, enables operators to obtain predictions about the expected remaining useful life. Numerous methods for HDD failure prediction have been published in recent years, and their evaluation has shown decent results. However, a naive splitting into training and test sets can lead to data leakage and, thus, over-optimistic results that cannot be achieved on the data of scientific interest. In this paper, we propose a novel data leakage measure for quantifying the amount of data leakage in training and test datasets. Further, we define four splitting techniques and evaluate our measure in terms of the performance optimism of classification models with respect to these different splitting strategies. Our results consistently show that splitting techniques prone to data leakage induce an overestimation of predictive performance. Overall, we were able to show the usefulness of the defined data leakage measure, as well as its connection with different splitting techniques and the performance optimism of prediction models.}, address = {New York, NY, USA}, author = {Grillmeyer, Daniel and Hadry, Marius and Lesch, Veronika and Borst, Vanessa and Leppich, Robert and Bauer, Andr\'{e} and Kounev, Samuel}, booktitle = {Proceedings of the 16th ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {334–346}, publisher = {Association for Computing Machinery}, series = {ICPE '25}, title = {Quantifying Data Leakage in Failure Prediction Tasks}, year = 2025 }
Evaluating the Benefits of Model Retraining for Self-Aware Vehicle Traffic Forecasting. Hadry, Marius; Grillmeyer, Daniel; Becker, Martin; König, Maximilian; Lesch, Veronika; Kounev, Samuel; in IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS) (2024). 41–50.
- [ BibTeX ]
- [ DOI ]
@inproceedings{10771392, author = {Hadry, Marius and Grillmeyer, Daniel and Becker, Martin and König, Maximilian and Lesch, Veronika and Kounev, Samuel}, booktitle = {IEEE International Conference on Autonomic Computing and Self-Organizing Systems (ACSOS)}, keywords = {descartes}, pages = {41-50}, title = {Evaluating the Benefits of Model Retraining for Self-Aware Vehicle Traffic Forecasting}, year = 2024 }

Short Papers

Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis. Bauer, André; Dittus, Timo; Straesser, Martin; Kamatar, Alok; Baughman, Matt; Beierlieb, Lukas; Hadry, Marius; Grillmeyer, Daniel; Lubas, Yannik; Kounev, Samuel; Foster, Ian; Chard, Kyle; in Companion of the 15th ACM/SPEC International Conference on Performance Engineering (2024). 72–76. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ Download ]
As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.

@inproceedings{10.1145/3629527.3651843, abstract = {As the market for cloud computing continues to grow, an increasing number of users are deploying applications as microservices. The shift introduces unique challenges in identifying and addressing performance issues, particularly within large and complex infrastructures. To address this challenge, we propose a methodology that unveils temporal performance deviations in microservices by clustering containers based on their performance characteristics at different time intervals. Showcasing our methodology on the Alibaba dataset, we found both stable and dynamic performance patterns, providing a valuable tool for enhancing overall performance and reliability in modern application landscapes.}, address = {New York, NY, USA}, author = {Bauer, André and Dittus, Timo and Straesser, Martin and Kamatar, Alok and Baughman, Matt and Beierlieb, Lukas and Hadry, Marius and Grillmeyer, Daniel and Lubas, Yannik and Kounev, Samuel and Foster, Ian and Chard, Kyle}, booktitle = {Companion of the 15th ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {72–76}, publisher = {Association for Computing Machinery}, series = {ICPE '24 Companion}, title = {Unveiling Temporal Performance Deviation: Leveraging Clustering in Microservices Performance Analysis}, year = 2024 }

Workshop Papers

Vision and Position Papers

Poster Papers

Tutorial Papers

Talks

Techreports