Publications

[ Conference ]
[ Inproceedings ]
[ Mastersthesis ]

Conference[ to top ]

Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning Bauer, André; Straesser, Martin; Beierlieb, Lukas; Meissner, Maximilian; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022).
- [ BibTeX ]
- [ EndNote ]
- [ Download ]
@conference{bauer2022automated, author = {Bauer, André and Straesser, Martin and Beierlieb, Lukas and Meissner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {timeseries}, month = 4, publisher = {ACM}, title = {Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning}, year = 2022 }

%0 Generic %1 bauer2022automated %A Bauer, André %A Straesser, Martin %A Beierlieb, Lukas %A Meissner, Maximilian %A Kounev, Samuel %B Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering %D 2022 %I ACM %T Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning

Inproceedings[ to top ]

Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs Bauer, Andr\’{e}; Straesser, Martin; Leznik, Mark; Beierlieb, Lukas; Hadry, Marius; Hudson, Nathaniel; Chard, Kyle; Kounev, Samuel; Foster, Ian; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 95–99.
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ URL ]
- [ Download ]
Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.

@inproceedings{10.1145/3578245.3584693, abstract = {Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.}, address = {New York, NY, USA}, author = {Bauer, Andr\'{e} and Straesser, Martin and Leznik, Mark and Beierlieb, Lukas and Hadry, Marius and Hudson, Nathaniel and Chard, Kyle and Kounev, Samuel and Foster, Ian}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {myown}, pages = {95–99}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs}, year = 2023 }

%0 Conference Paper %1 10.1145/3578245.3584693 %A Bauer, Andr\'{e} %A Straesser, Martin %A Leznik, Mark %A Beierlieb, Lukas %A Hadry, Marius %A Hudson, Nathaniel %A Chard, Kyle %A Kounev, Samuel %A Foster, Ian %B Companion of the 2023 ACM/SPEC International Conference on Performance Engineering %C New York, NY, USA %D 2023 %I Association for Computing Machinery %P 95–99 %T Searching for the Ground Truth: Assessing the Similarity of Benchmarking Runs %U https://doi.org/10.1145/3578245.3584693 %X Stable and repeatable measurements are essential for comparing the performance of different systems or applications, and benchmarks are used to ensure accuracy and replication. However, if the corresponding measurements are not stable and repeatable, wrong conclusions can be drawn. To facilitate the task of determining whether the measurements are similar, we used a data set of 586 micro-benchmarks to (i) analyze the data set itself, (ii) examine our previous approach, and (iii) propose and evaluate a heuristic. To evaluate the different approaches, we perform a peer review to assess the dissimilarity of the benchmark runs. Our results show that this task is challenging even for humans and that our heuristic exhibits a sensitivity of 92%.
Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes Prantl, Thomas; Zeck, Timo; Iffländer, Lukas; Beierlieb, Lukas; Dmitrenko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2022 5th Conference on Cloud and Internet of Things (CIoT) (2022).
- [ BibTeX ]
- [ EndNote ]
@inproceedings{PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark, author = {Prantl, Thomas and Zeck, Timo and Iffländer, Lukas and Beierlieb, Lukas and Dmitrenko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2022 5th Conference on Cloud and Internet of Things (CIoT)}, keywords = {myown}, organization = {IEEE}, title = {Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes}, year = 2022 }

%0 Conference Paper %1 PrZeIfBeDmKrKo2022-CIoT-ABEBenchmark %A Prantl, Thomas %A Zeck, Timo %A Iffländer, Lukas %A Beierlieb, Lukas %A Dmitrenko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2022 5th Conference on Cloud and Internet of Things (CIoT) %D 2022 %T Towards a Cryptography Benchmark: A View on Attribute Based Encryption Schemes
Software Testing Strategies for Detecting Hypercall Handlers{\textquotesingle} Aging-related Bugs Beierlieb, Lukas; Avritzer, Alberto; Ifflander, Lukas; Antunes, Nuno; Milenkoski, Aleksandar; Kounev, Samuel; in 2021 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW}) (2021).
- [ BibTeX ]
- [ EndNote ]
@inproceedings{Beierlieb_2021_SoftwareTestingStrategiesforDetectingHypercallHandlersAgingRelatedBugs, author = {Beierlieb, Lukas and Avritzer, Alberto and Ifflander, Lukas and Antunes, Nuno and Milenkoski, Aleksandar and Kounev, Samuel}, booktitle = {2021 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})}, keywords = {myown}, publisher = {{IEEE}}, title = {Software Testing Strategies for Detecting Hypercall Handlers{\textquotesingle} Aging-related Bugs}, year = 2021 }

%0 Conference Paper %1 Beierlieb_2021_SoftwareTestingStrategiesforDetectingHypercallHandlersAgingRelatedBugs %A Beierlieb, Lukas %A Avritzer, Alberto %A Ifflander, Lukas %A Antunes, Nuno %A Milenkoski, Aleksandar %A Kounev, Samuel %B 2021 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW}) %D 2021 %I {IEEE} %T Software Testing Strategies for Detecting Hypercall Handlers{\textquotesingle} Aging-related Bugs
Teaching Software Testing Using Automated Grading Beierlieb, Lukas; Iffländer, Lukas; Schneider, Tobias; Prantl, Thomas; Kounev, Samuel; in Proceedings of the Fifth Workshop "Automatische Bewertung von Programmieraufgaben" (ABP 2021),virtual event, October 28-29, 2021, A. Greubel, S. Strickroth, M. Striewe (Hrsg.) (2021).
- [ BibTeX ]
- [ EndNote ]
@inproceedings{Beierlieb_2021_TeachingSoftwareTestingUsingAutomatedGrading, author = {Beierlieb, Lukas and Iffländer, Lukas and Schneider, Tobias and Prantl, Thomas and Kounev, Samuel}, booktitle = {Proceedings of the Fifth Workshop "Automatische Bewertung von Programmieraufgaben" (ABP 2021),virtual event, October 28-29, 2021}, editor = {Greubel, André and Strickroth, Sven and Striewe, Michael}, keywords = {myown}, publisher = {Gesellschaft für Informatik}, title = {Teaching Software Testing Using Automated Grading}, year = 2021 }

%0 Conference Paper %1 Beierlieb_2021_TeachingSoftwareTestingUsingAutomatedGrading %A Beierlieb, Lukas %A Iffländer, Lukas %A Schneider, Tobias %A Prantl, Thomas %A Kounev, Samuel %B Proceedings of the Fifth Workshop "Automatische Bewertung von Programmieraufgaben" (ABP 2021),virtual event, October 28-29, 2021 %D 2021 %E Greubel, André %E Strickroth, Sven %E Striewe, Michael %I Gesellschaft für Informatik %T Teaching Software Testing Using Automated Grading
Measuring the Performance Impact of Branching Instructions Beierlieb, Lukas; Iffländer, Lukas; Milenkoski, Aleksandar; Prantl, Thomas; Kounev, Samuel; in Proceedings of the 12th Symposium on Software Performance 2021 (SSP’21) (2021).
- [ BibTeX ]
- [ EndNote ]
@inproceedings{Beierlieb_2021_MeasuringthePerformanceImpactofBranchingInstructions., author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Prantl, Thomas and Kounev, Samuel}, booktitle = {Proceedings of the 12th Symposium on Software Performance 2021 (SSP'21)}, keywords = {myown}, month = {nov}, title = {Measuring the Performance Impact of Branching Instructions}, year = 2021 }

%0 Conference Paper %1 Beierlieb_2021_MeasuringthePerformanceImpactofBranchingInstructions. %A Beierlieb, Lukas %A Iffländer, Lukas %A Milenkoski, Aleksandar %A Prantl, Thomas %A Kounev, Samuel %B Proceedings of the 12th Symposium on Software Performance 2021 (SSP'21) %D 2021 %T Measuring the Performance Impact of Branching Instructions
Performance Evaluation of a Post-Quantum Public-Key Cryptosystem Prantl, Thomas; Prantl, Dominik; Beierlieb, Lukas; Iffländer, Lukas; Dmitrienko, Alexandra; Krupitzer, Christian; Kounev, Samuel; in 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) (2021).
- [ BibTeX ]
- [ EndNote ]
@inproceedings{PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem, author = {Prantl, Thomas and Prantl, Dominik and Beierlieb, Lukas and Iffländer, Lukas and Dmitrienko, Alexandra and Krupitzer, Christian and Kounev, Samuel}, booktitle = {2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC)}, keywords = {myown}, organization = {IEEE}, title = {Performance Evaluation of a Post-Quantum Public-Key Cryptosystem}, year = 2021 }

%0 Conference Paper %1 PrPrBeIfDnKrKo2021-IPCCC-QuantumPublicKeyCryposystem %A Prantl, Thomas %A Prantl, Dominik %A Beierlieb, Lukas %A Iffländer, Lukas %A Dmitrienko, Alexandra %A Krupitzer, Christian %A Kounev, Samuel %B 2021 IEEE 40th International Performance Computing and Communications Conference (IPCCC) %D 2021 %T Performance Evaluation of a Post-Quantum Public-Key Cryptosystem
Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior Beierlieb, Lukas; Iffländer, Lukas; Kounev, Samuel; Milenkoski, Aleksandar; in Proceedings of the 10th Symposium on Software Performance 2019 (SSP’19) (2019).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ Download ]
With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.

@inproceedings{Beierlieb_2019_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior, abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.}, author = {Beierlieb, Lukas and Iffländer, Lukas and Kounev, Samuel and Milenkoski, Aleksandar}, booktitle = {Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)}, keywords = {t_workshop}, month = 11, title = {Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior}, year = 2019 }

%0 Conference Paper %1 Beierlieb_2019_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior %A Beierlieb, Lukas %A Iffländer, Lukas %A Kounev, Samuel %A Milenkoski, Aleksandar %B Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19) %D 2019 %T Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior %X With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.
Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces Beierlieb, Lukas; Iffländer, Lukas; Milenkoski, Aleksandar; Goncalves, Charles F.; Antunes, Nuno; Kounev, Samuel; in 2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW}) (2019).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ Download ]
With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.

@inproceedings{Beierlieb_2019_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces, abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.}, author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Goncalves, Charles F. and Antunes, Nuno and Kounev, Samuel}, booktitle = {2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})}, keywords = {t_workshop}, month = 11, organization = {{IEEE}}, title = {Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces}, year = 2019 }

%0 Conference Paper %1 Beierlieb_2019_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces %A Beierlieb, Lukas %A Iffländer, Lukas %A Milenkoski, Aleksandar %A Goncalves, Charles F. %A Antunes, Nuno %A Kounev, Samuel %B 2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW}) %D 2019 %T Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces %X With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.
Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking Iffländer, Lukas; Geißler, Stefan; Walter, Jürgen; Beierlieb, Lukas; Kounev, Samuel; in Proceedings of the 9th Symposium on Software Performance 2018 (SSP’18) (2018).
- [ Abstract ]
- [ BibTeX ]
- [ EndNote ]
- [ Download ]
Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Mooreâ€™s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

@inproceedings{Ifflaender2018_AddressingShortcomingsofExistingDDoSProtectionSoftwareUsingSoftware-DefinedNetworking, abstract = {Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Mooreâ€™s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.}, author = {Iffländer, Lukas and Geißler, Stefan and Walter, Jürgen and Beierlieb, Lukas and Kounev, Samuel}, booktitle = {Proceedings of the 9th Symposium on Software Performance 2018 (SSP'18)}, keywords = {t_workshop}, month = {November}, title = {Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking}, year = 2018 }

%0 Conference Paper %1 Ifflaender2018_AddressingShortcomingsofExistingDDoSProtectionSoftwareUsingSoftware-DefinedNetworking %A Iffländer, Lukas %A Geißler, Stefan %A Walter, Jürgen %A Beierlieb, Lukas %A Kounev, Samuel %B Proceedings of the 9th Symposium on Software Performance 2018 (SSP'18) %D 2018 %T Addressing Shortcomings of Existing DDoS Protection Software Using Software-Defined Networking %X Attacks on software systems are becoming more and more frequent, aggressive and sophisticated. With the changing threat landscape, in 2018, organizations are looking at when they will be attacked, not if. Intrusion Detection Systems (IDSs) can help in defending against these attacks. The systems that host IDSs require extensive computing resources as IDSs tend to detect attacks under overloaded conditions wrongfully. With the end of Mooreâ€™s law and the growing adoption of Internet of Things, designers of security systems can no longer expect processing power to keep up the pace with them. This limitation requires ways to increase the performance of these systems without adding additional compute power. In this work, we present two dynamic and a static approach to bypass IDS for traffic deemed benign. We provide its prototype implementation and evaluate our solution. Our evaluation shows promising results. Performance is increased up to the level of a system without an IDS. Attack detection is within the margin of error from the 100% rate. However, our findings show that dynamic approaches perform best when using software switches. The use of a hardware switch reduces the detection rate and performance significantly.

Mastersthesis[ to top ]

Hypercall Interface Testing for the Example of Hyper-V Beierlieb, Lukas; Thesis; University of Würzburg; Am Hubland, Informatikgebäude, 97074 Würzburg, Germany. (2019, Dezember).

Würzburg Software Engineering Award Runner Up Masterthesis
- [ BibTeX ]
- [ EndNote ]
- [ Download ]
@mastersthesis{beierlieb2019hypercall, address = {Am Hubland, Informatikgebäude, 97074 Würzburg, Germany}, author = {Beierlieb, Lukas}, keywords = {myown}, month = {december}, note = {Würzburg Software Engineering Award Runner Up Masterthesis}, school = {University of Würzburg}, title = {Hypercall Interface Testing for the Example of Hyper-V}, type = {Bachelor Thesis}, year = 2019 }

%0 Thesis %1 beierlieb2019hypercall %A Beierlieb, Lukas %C Am Hubland, Informatikgebäude, 97074 Würzburg, Germany %D 2019 %T Hypercall Interface Testing for the Example of Hyper-V
Performance and Security Influence of Augmenting DDoS Protection Systems using SDN and NFV Beierlieb, Lukas; Thesis; University of W{\"u}rzburg; Am Hubland, Informatikgeb{\"a}ude, 97074 W{\"u}rzburg, Germany. (2018, Januar).

Würzburg Software Engineering Award sponsored by Infosim GmbH & Co. KG
- [ BibTeX ]
- [ EndNote ]
- [ Download ]
@mastersthesis{beierlieb2018ba, address = {Am Hubland, Informatikgeb{\"a}ude, 97074 W{\"u}rzburg, Germany}, author = {Beierlieb, Lukas}, keywords = {myown}, month = {January}, note = {Würzburg Software Engineering Award sponsored by Infosim GmbH & Co. KG}, school = {University of W{\"u}rzburg}, title = {Performance and Security Influence of Augmenting DDoS Protection Systems using SDN and NFV}, type = {Bachelor Thesis}, year = 2018 }

%0 Thesis %1 beierlieb2018ba %A Beierlieb, Lukas %C Am Hubland, Informatikgeb{\"a}ude, 97074 W{\"u}rzburg, Germany %D 2018 %T Performance and Security Influence of Augmenting DDoS Protection Systems using SDN and NFV

Publications

Data privacy protection

Data privacy protection

Picture credits