Publications

Journal and Magazine Articles

Full Papers

Short Papers

Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning. Bauer, André; Straesser, Martin; Beierlieb, Lukas; Meissner, Maximilian; Kounev, Samuel; in Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering (2022). ACM.
- [ BibTeX ]
- [ Download ]
@conference{bauer2022automated, author = {Bauer, André and Straesser, Martin and Beierlieb, Lukas and Meissner, Maximilian and Kounev, Samuel}, booktitle = {Proceedings of the 2022 ACM/SPEC on International Conference on Performance Engineering}, keywords = {machine_learning}, month = {04}, publisher = {ACM}, title = {Automated Triage of Performance Change Points Using Time Series Analysis and Machine Learning}, year = 2022 }

Workshop Papers

Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency. Meissner, Maximilian; Kamthania, Supriya; Rawtani, Nishant; Bucek, James; Lange, Klaus-Dieter; Kounev, Samuel; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022). ACM, New York, NY, USA.
- [ BibTeX ]
- [ Download ]
@inproceedings{meissner2022experience, address = {New York, NY, USA}, author = {Meissner, Maximilian and Kamthania, Supriya and Rawtani, Nishant and Bucek, James and Lange, Klaus-Dieter and Kounev, Samuel}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {t_workshop}, month = {04}, publisher = {ACM}, title = {Experience and Guidelines for Sorting Algorithm Choices and Their Energy Efficiency}, year = 2022 }

Book Chapters

Server Energy Efficiency Benchmarks. von Kistowski, J’oakim; Meißner, Maximilian; Lange, Klaus-Dieter; Arnold, Jeremy A. In Systems Benchmarking: For Scientists and Engineers, pp. 247–265. Springer Nature Switzerland, Cham, 2025.
- [ Abstract ]
- [ BibTeX ]
- [ DOI ]
This chapter starts by introducing the fundamentals of power and efficiency measurements and then describes a power and performance benchmark methodology developed by the SPECpower committee for commodity servers. It is designed to characterize and rate the energy efficiency of a system under test for multiple load levels, showcasing load level differences in system behavior regarding energy efficiency. The methodology was first implemented in the SPECpower_ssj 2008 benchmark and later extended with more workloads, metrics, and other application areas for the SPEC Server Efficiency Rating Tool (SERT) suite. The SERT suite was developed to fill the need for a rating tool that can be utilized by government agencies in their regulatory programs, for example, the U.S. Environmental Protection Agency (EPA) for the use in their ENERGY STAR program for servers.

@incollection{sbBookSecondEd, abstract = {This chapter starts by introducing the fundamentals of power and efficiency measurements and then describes a power and performance benchmark methodology developed by the SPECpower committee for commodity servers. It is designed to characterize and rate the energy efficiency of a system under test for multiple load levels, showcasing load level differences in system behavior regarding energy efficiency. The methodology was first implemented in the SPECpower{_}ssj 2008 benchmark and later extended with more workloads, metrics, and other application areas for the SPEC Server Efficiency Rating Tool (SERT) suite. The SERT suite was developed to fill the need for a rating tool that can be utilized by government agencies in their regulatory programs, for example, the U.S. Environmental Protection Agency (EPA) for the use in their ENERGY STAR program for servers.}, address = {Cham}, author = {von Kistowski, J{'o}akim and Mei{\ss}ner, Maximilian and Lange, Klaus-Dieter and Arnold, Jeremy A.}, booktitle = {Systems Benchmarking: For Scientists and Engineers}, keywords = {t_bookchapter}, pages = {247–265}, publisher = {Springer Nature Switzerland}, title = {Server Energy Efficiency Benchmarks}, year = 2025 }

Vision and Position Papers

Challenges and Future Directions in Efficiency Benchmarking (Vision Paper). Meissner, Maximilian; Lange, Klaus-Dieter; Sharma, Sanjay; Arnold, Jeremy; Cragin, Aaron; Galizia, Patrick; Petrich, Mike; Zhang, Bin; Kounev, Samuel; in Companion of the 2023 ACM/SPEC International Conference on Performance Engineering (2023). 51–55. Association for Computing Machinery, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
SPEC benchmarks are crucial contributors behind the improvement of server efficiency since 2007, given their role in making the power consumption and efficiency of servers transparent for government regulators, customers, and the manufacturers themselves.As the IT landscape experiences radical transformations, efficiency benchmarks need to be updated accordingly to generate results relevant to government regulators, manufacturers, and customers. In this paper, we outline current challenges efficiency benchmark developers are tackling and highlight recent technological developments the next generation of efficiency benchmarks should take into account.

@inproceedings{10.1145/3578245.3585034, abstract = {SPEC benchmarks are crucial contributors behind the improvement of server efficiency since 2007, given their role in making the power consumption and efficiency of servers transparent for government regulators, customers, and the manufacturers themselves.As the IT landscape experiences radical transformations, efficiency benchmarks need to be updated accordingly to generate results relevant to government regulators, manufacturers, and customers. In this paper, we outline current challenges efficiency benchmark developers are tackling and highlight recent technological developments the next generation of efficiency benchmarks should take into account.}, address = {New York, NY, USA}, author = {Meissner, Maximilian and Lange, Klaus-Dieter and Sharma, Sanjay and Arnold, Jeremy and Cragin, Aaron and Galizia, Patrick and Petrich, Mike and Zhang, Bin and Kounev, Samuel}, booktitle = {Companion of the 2023 ACM/SPEC International Conference on Performance Engineering}, keywords = {descartes}, pages = {51–55}, publisher = {Association for Computing Machinery}, series = {ICPE '23 Companion}, title = {Challenges and Future Directions in Efficiency Benchmarking (Vision Paper)}, year = 2023 }

Poster Papers

Tutorial Papers

Next Generation Energy Efficiency Benchmarking: Reliable and Reproducible Efficiency Measurements in a Diverse IT-Landscape. Meissner, Maximilian; Lange, Klaus-Dieter; Cragin, Aaron; H J, Pushpa; Proulx, Robert; Kounev, Samuel; in Companion of the 16th ACM/SPEC International Conference on Performance Engineering (2025). 60–64. Association for Computing Machinery, Toronto ON, Canada.
- [ Abstract ]
- [ BibTeX ]
- [ URL ]
- [ DOI ]
Energy efficiency has become a critical quality criterion of IT-systems. While hardware designers have long worked on improving efficiency, researchers in other fields, such as software design and Artificial Intelligence, are increasingly focusing on this issue as well. Such combined efforts are essential to mitigate the significant energy demands of current and future hardware and software. In comparison to performance benchmarking, energy efficiency benchmarking involves a variety of additional factors that must be carefully considered and controlled. Without proper setup and oversight, measurements can easily lead to misleading, inconclusive, or unreliable results. Because reproducibility of experiment results, or lack thereof, is a critical concern in the scientific community, knowing the intricacies of power and efficiency measurements is a crucial pre-requisite for carrying out meaningful experiments. This paper offers insight into aspects important for conducting energy efficiency experiments based on best practices. It explains the effects of subtle differences in experiment setup and discusses current trends and challenges in efficiency benchmarking.

@inproceedings{10.1145/3680256.3721255, abstract = {Energy efficiency has become a critical quality criterion of IT-systems. While hardware designers have long worked on improving efficiency, researchers in other fields, such as software design and Artificial Intelligence, are increasingly focusing on this issue as well. Such combined efforts are essential to mitigate the significant energy demands of current and future hardware and software. In comparison to performance benchmarking, energy efficiency benchmarking involves a variety of additional factors that must be carefully considered and controlled. Without proper setup and oversight, measurements can easily lead to misleading, inconclusive, or unreliable results. Because reproducibility of experiment results, or lack thereof, is a critical concern in the scientific community, knowing the intricacies of power and efficiency measurements is a crucial pre-requisite for carrying out meaningful experiments. This paper offers insight into aspects important for conducting energy efficiency experiments based on best practices. It explains the effects of subtle differences in experiment setup and discusses current trends and challenges in efficiency benchmarking.}, address = {New York, NY, USA}, author = {Meissner, Maximilian and Lange, Klaus-Dieter and Cragin, Aaron and H J, Pushpa and Proulx, Robert and Kounev, Samuel}, booktitle = {Companion of the 16th ACM/SPEC International Conference on Performance Engineering}, keywords = {t_tutorial}, pages = {60–64}, publisher = {Association for Computing Machinery}, series = {ICPE '25}, title = {Next Generation Energy Efficiency Benchmarking: Reliable and Reproducible Efficiency Measurements in a Diverse IT-Landscape}, year = 2025 }
SPEC Efficiency Benchmark Development: How to Contribute to the Future of Energy Conservation. Meissner, Maximilian; Lange, Klaus-Dieter; Arnold, Jeremy; Sharma, Sanjay; Tipley, Roger; Rawtani, Nishant; Reiner, David; Petrich, Mike; Cragin, Aaron; in Companion of the 2022 ACM/SPEC International Conference on Performance Engineering (2022). ACM, New York, NY, USA.
- [ Abstract ]
- [ BibTeX ]
- [ Download ]
A driving force behind the improvement of server efficiency in recent years is the use of SPEC benchmarks. They are used in mandatory government regulations, the ISO/IEC 21836:2020 standard, and product marketing, giving server manufacturers and buyers significant incentive to improve energy efficiency. To produce relevant results, benchmarks need to take into account future trends in hardware and software development, such as the introduction of new accelerators and workloads. To keep pace with the development of the fast moving IT landscape, SPEC plans to introduce a workload bounty program to encourage researchers to develop novel workloads. Submitted workloads will be considered for inclusion in future SPEC Efficiency benchmarks and rewarded. In this paper, we outline the process of energy-efficiency benchmark development. SPEC ensures the development of high-quality benchmarks for government regulations through its extensive experience and collaboration with stakeholders from industry, academia, and governments. One of the tools that emerged from this process is the Chauffeur Worklet Development Kit (WDK), which can be used by researchers to develop next-generation workloads to enhance the real-world relevance of future SPEC benchmarks, a critical element for the benchmarks to contribute to future energy conservation.

@inproceedings{meissner2022efficiency, abstract = {A driving force behind the improvement of server efficiency in recent years is the use of SPEC benchmarks. They are used in mandatory government regulations, the ISO/IEC 21836:2020 standard, and product marketing, giving server manufacturers and buyers significant incentive to improve energy efficiency. To produce relevant results, benchmarks need to take into account future trends in hardware and software development, such as the introduction of new accelerators and workloads. To keep pace with the development of the fast moving IT landscape, SPEC plans to introduce a workload bounty program to encourage researchers to develop novel workloads. Submitted workloads will be considered for inclusion in future SPEC Efficiency benchmarks and rewarded. In this paper, we outline the process of energy-efficiency benchmark development. SPEC ensures the development of high-quality benchmarks for government regulations through its extensive experience and collaboration with stakeholders from industry, academia, and governments. One of the tools that emerged from this process is the Chauffeur Worklet Development Kit (WDK), which can be used by researchers to develop next-generation workloads to enhance the real-world relevance of future SPEC benchmarks, a critical element for the benchmarks to contribute to future energy conservation.}, address = {New York, NY, USA}, author = {Meissner, Maximilian and Lange, Klaus-Dieter and Arnold, Jeremy and Sharma, Sanjay and Tipley, Roger and Rawtani, Nishant and Reiner, David and Petrich, Mike and Cragin, Aaron}, booktitle = {Companion of the 2022 ACM/SPEC International Conference on Performance Engineering}, keywords = {spec}, month = {04}, publisher = {ACM}, title = {SPEC Efficiency Benchmark Development: How to Contribute to the Future of Energy Conservation}, year = 2022 }

Journal and Magazine Articles

Full Papers

Short Papers

Workshop Papers

Book Chapters

Vision and Position Papers

Poster Papers

Tutorial Papers

Talks

Techreports

Bildnachweise