Publications

Measuring the Performance Impact of Branching Instructions. Beierlieb, Lukas; Iffländer, Lukas; Milenkoski, Aleksandar; Prantl, Thomas; Kounev, Samuel; in Proceedings of the 12th Symposium on Software Performance 2021 (SSP’21) (2021).

[ BibTeX ]
[ EndNote ]

@inproceedings{Beierlieb_2021_MeasuringthePerformanceImpactofBranchingInstructions.,
  author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Prantl, Thomas and Kounev, Samuel},
  booktitle = {Proceedings of the 12th Symposium on Software Performance 2021 (SSP'21)},
  keywords = {sys:relevantfor:se-group},
  month = 11,
  title = {Measuring the Performance Impact of Branching Instructions},
  year = 2021
}

%0 Conference Paper
%1 Beierlieb_2021_MeasuringthePerformanceImpactofBranchingInstructions.
%A Beierlieb, Lukas
%A Iffländer, Lukas
%A Milenkoski, Aleksandar
%A Prantl, Thomas
%A Kounev, Samuel
%B Proceedings of the 12th Symposium on Software Performance 2021 (SSP'21)
%D 2021
%T Measuring the Performance Impact of Branching Instructions

Software Testing Strategies for Detecting Hypercall Handlers{\textquotesingle} Aging-related Bugs. Beierlieb, Lukas; Avritzer, Alberto; Ifflander, Lukas; Antunes, Nuno; Milenkoski, Aleksandar; Kounev, Samuel; in 2021 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW}) (2021). {IEEE}.

[ BibTeX ]
[ EndNote ]

@inproceedings{Beierlieb_2021_SoftwareTestingStrategiesforDetectingHypercallHandlersAgingRelatedBugs,
  author = {Beierlieb, Lukas and Avritzer, Alberto and Ifflander, Lukas and Antunes, Nuno and Milenkoski, Aleksandar and Kounev, Samuel},
  booktitle = {2021 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})},
  keywords = {security},
  publisher = {{IEEE}},
  title = {Software Testing Strategies for Detecting Hypercall Handlers{\textquotesingle} Aging-related Bugs},
  year = 2021
}

%0 Conference Paper
%1 Beierlieb_2021_SoftwareTestingStrategiesforDetectingHypercallHandlersAgingRelatedBugs
%A Beierlieb, Lukas
%A Avritzer, Alberto
%A Ifflander, Lukas
%A Antunes, Nuno
%A Milenkoski, Aleksandar
%A Kounev, Samuel
%B 2021 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})
%D 2021
%I {IEEE}
%T Software Testing Strategies for Detecting Hypercall Handlers{\textquotesingle} Aging-related Bugs

Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces. Beierlieb, Lukas; Iffländer, Lukas; Milenkoski, Aleksandar; Goncalves, Charles F.; Antunes, Nuno; Kounev, Samuel; in 2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW}) (2019). {IEEE}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.

@inproceedings{Beierlieb_2019_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces,
  abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.},
  author = {Beierlieb, Lukas and Iffländer, Lukas and Milenkoski, Aleksandar and Goncalves, Charles F. and Antunes, Nuno and Kounev, Samuel},
  booktitle = {2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})},
  keywords = {t_workshop},
  month = 11,
  organization = {{IEEE}},
  title = {Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces},
  year = 2019
}

%0 Conference Paper
%1 Beierlieb_2019_TowardsTestingtheSoftwareAgingBehaviorofHypervisorHypercallInterfaces
%A Beierlieb, Lukas
%A Iffländer, Lukas
%A Milenkoski, Aleksandar
%A Goncalves, Charles F.
%A Antunes, Nuno
%A Kounev, Samuel
%B 2019 {IEEE} International Symposium on Software Reliability Engineering Workshops ({ISSREW})
%D 2019
%T Towards Testing the Software Aging Behavior of Hypervisor Hypercall Interfaces
%X With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. As long-running applications, they are susceptible to software aging. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research either deals with the aging properties of hypervisors in general without considering the hypercalls or focuses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports extensive test campaigns as well as the modeling of hypercall interfaces.

Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior. Beierlieb, Lukas; Iffländer, Lukas; Kounev, Samuel; Milenkoski, Aleksandar; in Proceedings of the 10th Symposium on Software Performance 2019 (SSP’19) (2019).

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ Download ]

With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.

@inproceedings{Beierlieb_2019_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior,
  abstract = {With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.},
  author = {Beierlieb, Lukas and Iffländer, Lukas and Kounev, Samuel and Milenkoski, Aleksandar},
  booktitle = {Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)},
  keywords = {t_workshop},
  month = 11,
  title = {Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior},
  year = 2019
}

%0 Conference Paper
%1 Beierlieb_2019_TowardsTestingthePerformanceInfluenceofHypervisorHypercallInterfaceBehavior
%A Beierlieb, Lukas
%A Iffländer, Lukas
%A Kounev, Samuel
%A Milenkoski, Aleksandar
%B Proceedings of the 10th Symposium on Software Performance 2019 (SSP'19)
%D 2019
%T Towards Testing the Performance Influence of Hypervisor Hypercall Interface Behavior
%X With the continuing rise of cloud technology hypervisors play a vital role in the performance and reliability of current services. Hypervisors offer so-called hypercall interfaces for communication with the hosted virtual machines. These interfaces require thorough robustness to assure performance, security, and reliability. Existing research focusses on finding hypercall-related vulnerabilities. In this work, we discuss open challenges regarding hypercall interfaces. To address these challenges, we propose an extensive framework architecture to perform robustness testing on hypercall interfaces. This framework supports test campaigns and modeling of hypercall interfaces.

{CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts}. Milenkoski, Aleksandar; Iosup, Alexandru; Kounev, Samuel; Sachs, Kai; Mularz, Diane E.; Curtiss, Jonathan A.; Ding, Jason J.; Rosenberg, Florian; Rygielski, Piotr; in IEEE Cloud Computing (2018). 5(3) 65–76. IEEE.

[ BibTeX ]
[ EndNote ]

@article{MiIoKoSaMuCuDiRoRy2018-IEEECC-cup,
  author = {Milenkoski, Aleksandar and Iosup, Alexandru and Kounev, Samuel and Sachs, Kai and Mularz, Diane E. and Curtiss, Jonathan A. and Ding, Jason J. and Rosenberg, Florian and Rygielski, Piotr},
  journal = {IEEE Cloud Computing},
  keywords = {Virtualization},
  month = {06},
  number = 3,
  pages = {65–76},
  publisher = {IEEE},
  title = {{CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts}},
  volume = 5,
  year = 2018
}

%0 Journal Article
%1 MiIoKoSaMuCuDiRoRy2018-IEEECC-cup
%A Milenkoski, Aleksandar
%A Iosup, Alexandru
%A Kounev, Samuel
%A Sachs, Kai
%A Mularz, Diane E.
%A Curtiss, Jonathan A.
%A Ding, Jason J.
%A Rosenberg, Florian
%A Rygielski, Piotr
%D 2018
%I IEEE
%J IEEE Cloud Computing
%N 3
%P 65--76
%T {CUP: A Formalism for Expressing Cloud Usage Patterns for Experts and Non-Experts}
%V 5

{Benchmarking Intrusion Detection Systems with Adaptive Provisioning of Virtualized Resources}. Milenkoski, Aleksandar; Jayaram, K. R.; Kounev, Samuel; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

With the increasing popularity of virtualization, deploying intrusion detection systems (IDSes) in virtualized environments, for example, in virtual machines as virtualized network functions, has become an emerging practice. Modern virtualized environments feature on demand provisioning of virtualized processing and memory resources to virtual machines, dynamically adapting its intensity in order to meet resource demands. Such a provisioning may have a significant impact on many properties of an IDS deployed in a virtual machine, for example, on its attack detection accuracy. However, conventional metrics for quantifying IDS attack detection accuracy do not capture this impact, which may lead to inaccurate assessments of the IDSâ€™s accuracy at detecting attacks. In this chapter, we discuss in detail on the impact of on demand provisioning of virtualized resources on IDS attack detection accuracy. Further, we discuss on relevant issues related to the use of conventional metrics for quantifying IDS attack detection accuracy. Finally, we present a preliminary metric and measurement methodologies, which allow for the accurate assessment of IDS attack detection accuracy taking on-demand resource provisioning into account.

@incollection{MiJaKo2017-self,
  abstract = {With the increasing popularity of virtualization, deploying intrusion detection systems (IDSes) in virtualized environments, for example, in virtual machines as virtualized network functions, has become an emerging practice. Modern virtualized environments feature on demand provisioning of virtualized processing and memory resources to virtual machines, dynamically adapting its intensity in order to meet resource demands. Such a provisioning may have a significant impact on many properties of an IDS deployed in a virtual machine, for example, on its attack detection accuracy. However, conventional metrics for quantifying IDS attack detection accuracy do not capture this impact, which may lead to inaccurate assessments of the IDSâ€™s accuracy at detecting attacks. In this chapter, we discuss in detail on the impact of on demand provisioning of virtualized resources on IDS attack detection accuracy. Further, we discuss on relevant issues related to the use of conventional metrics for quantifying IDS attack detection accuracy. Finally, we present a preliminary metric and measurement methodologies, which allow for the accurate assessment of IDS attack detection accuracy taking on-demand resource provisioning into account.},
  address = {{Berlin Heidelberg, Germany}},
  author = {Milenkoski, Aleksandar and Jayaram, K. R. and Kounev, Samuel},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {t_bookchapter},
  publisher = {{Springer Verlag}},
  title = {{Benchmarking Intrusion Detection Systems with Adaptive Provisioning of Virtualized Resources}},
  year = 2017
}

%0 Book Section
%1 MiJaKo2017-self
%A Milenkoski, Aleksandar
%A Jayaram, K. R.
%A Kounev, Samuel
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Benchmarking Intrusion Detection Systems with Adaptive Provisioning of Virtualized Resources}
%U https://link.springer.com/chapter/10.1007%2F978-3-319-47474-8_22
%X With the increasing popularity of virtualization, deploying intrusion detection systems (IDSes) in virtualized environments, for example, in virtual machines as virtualized network functions, has become an emerging practice. Modern virtualized environments feature on demand provisioning of virtualized processing and memory resources to virtual machines, dynamically adapting its intensity in order to meet resource demands. Such a provisioning may have a significant impact on many properties of an IDS deployed in a virtual machine, for example, on its attack detection accuracy. However, conventional metrics for quantifying IDS attack detection accuracy do not capture this impact, which may lead to inaccurate assessments of the IDSâ€™s accuracy at detecting attacks. In this chapter, we discuss in detail on the impact of on demand provisioning of virtualized resources on IDS attack detection accuracy. Further, we discuss on relevant issues related to the use of conventional metrics for quantifying IDS attack detection accuracy. Finally, we present a preliminary metric and measurement methodologies, which allow for the accurate assessment of IDS attack detection accuracy taking on-demand resource provisioning into account.

{Software Architectures for Self-Protection in IaaS Clouds}. Jayaram, K. R.; Milenkoski, Aleksandar; Kounev, Samuel; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

In this chapter, we focus on software architectures for self-protection in IaaS clouds. IaaS clouds, especially hybrid clouds, are becoming increasingly popular because of the need for developers and enterprises to dynamically increase/decrease their use of computing resources to adapt quickly to market forces and customer demands, reduce costs, and increase fault tolerance. However, the adoption of public IaaS and hybrid clouds by enterprises is slower than expected because the current hybrid cloud infrastructures do not provide scalable and efficient mechanisms to prevent software tampering and configuration errors and ensure the trustworthiness and integrity of the software stack executing a hybrid application workload; or to enforce governmental privacy and audit regulations by ensuring that remote data and computation do not cross specified geographic boundaries. We discuss the recent research on integrating intrusion detection systems in IaaS infrastructures, as well as hardware-rooted integrity verification and geographic fencing to address the concerns outlined above.

@incollection{JaMiKo2017-self,
  abstract = {In this chapter, we focus on software architectures for self-protection in IaaS clouds. IaaS clouds, especially hybrid clouds, are becoming increasingly popular because of the need for developers and enterprises to dynamically increase/decrease their use of computing resources to adapt quickly to market forces and customer demands, reduce costs, and increase fault tolerance. However, the adoption of public IaaS and hybrid clouds by enterprises is slower than expected because the current hybrid cloud infrastructures do not provide scalable and efficient mechanisms to prevent software tampering and configuration errors and ensure the trustworthiness and integrity of the software stack executing a hybrid application workload; or to enforce governmental privacy and audit regulations by ensuring that remote data and computation do not cross specified geographic boundaries. We discuss the recent research on integrating intrusion detection systems in IaaS infrastructures, as well as hardware-rooted integrity verification and geographic fencing to address the concerns outlined above.},
  address = {{Berlin Heidelberg, Germany}},
  author = {Jayaram, K. R. and Milenkoski, Aleksandar and Kounev, Samuel},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {t_bookchapter},
  publisher = {{Springer Verlag}},
  title = {{Software Architectures for Self-Protection in IaaS Clouds}},
  year = 2017
}

%0 Book Section
%1 JaMiKo2017-self
%A Jayaram, K. R.
%A Milenkoski, Aleksandar
%A Kounev, Samuel
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Software Architectures for Self-Protection in IaaS Clouds}
%U https://link.springer.com/chapter/10.1007%2F978-3-319-47474-8_21
%X In this chapter, we focus on software architectures for self-protection in IaaS clouds. IaaS clouds, especially hybrid clouds, are becoming increasingly popular because of the need for developers and enterprises to dynamically increase/decrease their use of computing resources to adapt quickly to market forces and customer demands, reduce costs, and increase fault tolerance. However, the adoption of public IaaS and hybrid clouds by enterprises is slower than expected because the current hybrid cloud infrastructures do not provide scalable and efficient mechanisms to prevent software tampering and configuration errors and ensure the trustworthiness and integrity of the software stack executing a hybrid application workload; or to enforce governmental privacy and audit regulations by ensuring that remote data and computation do not cross specified geographic boundaries. We discuss the recent research on integrating intrusion detection systems in IaaS infrastructures, as well as hardware-rooted integrity verification and geographic fencing to address the concerns outlined above.

{Self-Aware Computing Systems} Kounev, Samuel; Kephart, Jeffrey O.; Milenkoski, Aleksandar; Zhu, Xiaoyun; (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ BibTeX ]
[ EndNote ]
[ URL ]
[ DOI ]

@book{KoKeMiZh2017-self,
  address = {{Berlin Heidelberg, Germany}},
  author = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {Self-adaptive-systems},
  publisher = {{Springer Verlag}},
  title = {{Self-Aware Computing Systems}},
  year = 2017
}

%0 Book
%1 KoKeMiZh2017-self
%A Kounev, Samuel
%A Kephart, Jeffrey O.
%A Milenkoski, Aleksandar
%A Zhu, Xiaoyun
%C {Berlin Heidelberg, Germany}
%D 2017
%I {Springer Verlag}
%R 10.1007/978-3-319-47474-8
%T {Self-Aware Computing Systems}
%U http://www.springer.com/de/book/9783319474724
%@ 978-3-319-47474-8

{Metrics and Benchmarks for Self-Aware Computing Systems}. Herbst, Nikolas; Becker, Steffen; Kounev, Samuel; Koziolek, Heiko; Maggio, Martina; Milenkoski, Aleksandar; Smirni, Evgenia; in {Self-Aware Computing Systems}, S. Kounev, J. O. Kephart, A. Milenkoski, X. Zhu (eds.) (2017). {Springer Verlag}, {Berlin Heidelberg, Germany}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

{In this chapter, we propose a list of metrics grouped by the MAPE-K paradigm for quantifying properties of self-aware computing systems. This set of metrics can be seen as a starting point toward benchmarking and comparing self-aware computing systems on a level-playing field. We discuss state-of-the art approaches in the related fields of self-adaptation and self-protection to identify commonalities in metrics for self-aware computing. We illustrate the need for benchmarking self-aware computing systems with the help of an approach that uncovers real-time characteristics of operating systems. Gained insights of this approach can be seen as a way of enhancing self-awareness by a measurement methodology on an ongoing basis. At the end of this chapter, we address new challenges in reference workload definition for benchmarking self-aware computing systems, namely load intensity patterns and burstiness modeling.}

@incollection{HeBeKoKoMaMiSm2017-self,
  abstract = {{In this chapter, we propose a list of metrics grouped by the MAPE-K paradigm for quantifying properties of self-aware computing systems. This set of metrics can be seen as a starting point toward benchmarking and comparing self-aware computing systems on a level-playing field. We discuss state-of-the art approaches in the related fields of self-adaptation and self-protection to identify commonalities in metrics for self-aware computing. We illustrate the need for benchmarking self-aware computing systems with the help of an approach that uncovers real-time characteristics of operating systems. Gained insights of this approach can be seen as a way of enhancing self-awareness by a measurement methodology on an ongoing basis. At the end of this chapter, we address new challenges in reference workload definition for benchmarking self-aware computing systems, namely load intensity patterns and burstiness modeling.}},
  address = {{Berlin Heidelberg, Germany}},
  author = {Herbst, Nikolas and Becker, Steffen and Kounev, Samuel and Koziolek, Heiko and Maggio, Martina and Milenkoski, Aleksandar and Smirni, Evgenia},
  booktitle = {{Self-Aware Computing Systems}},
  editor = {Kounev, Samuel and Kephart, Jeffrey O. and Milenkoski, Aleksandar and Zhu, Xiaoyun},
  keywords = {Self-adaptive-systems},
  publisher = {{Springer Verlag}},
  title = {{Metrics and Benchmarks for Self-Aware Computing Systems}},
  year = 2017
}

%0 Book Section
%1 HeBeKoKoMaMiSm2017-self
%A Herbst, Nikolas
%A Becker, Steffen
%A Kounev, Samuel
%A Koziolek, Heiko
%A Maggio, Martina
%A Milenkoski, Aleksandar
%A Smirni, Evgenia
%B {Self-Aware Computing Systems}
%C {Berlin Heidelberg, Germany}
%D 2017
%E Kounev, Samuel
%E Kephart, Jeffrey O.
%E Milenkoski, Aleksandar
%E Zhu, Xiaoyun
%I {Springer Verlag}
%T {Metrics and Benchmarks for Self-Aware Computing Systems}
%U https://link.springer.com/chapter/10.1007%2F978-3-319-47474-8_14
%X {In this chapter, we propose a list of metrics grouped by the MAPE-K paradigm for quantifying properties of self-aware computing systems. This set of metrics can be seen as a starting point toward benchmarking and comparing self-aware computing systems on a level-playing field. We discuss state-of-the art approaches in the related fields of self-adaptation and self-protection to identify commonalities in metrics for self-aware computing. We illustrate the need for benchmarking self-aware computing systems with the help of an approach that uncovers real-time characteristics of operating systems. Gained insights of this approach can be seen as a way of enhancing self-awareness by a measurement methodology on an ongoing basis. At the end of this chapter, we address new challenges in reference workload definition for benchmarking self-aware computing systems, namely load intensity patterns and burstiness modeling.}

Evaluation of Intrusion Detection Systems in Virtualized Environments. Milenkoski, Aleksandar; (2016, November). University of W{ü}rzburg, Germany.

{SPEC Kaivalya Dixit Distinguished Dissertation Award 2017}

[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

@phdthesis{Milenkoski2016-phd,
  author = {Milenkoski, Aleksandar},
  keywords = {descartes},
  month = 11,
  note = {{SPEC Kaivalya Dixit Distinguished Dissertation Award 2017}},
  school = {University of W{\"u}rzburg, Germany},
  title = {Evaluation of Intrusion Detection Systems in Virtualized Environments},
  year = 2016
}

%0 Thesis
%1 Milenkoski2016-phd
%A Milenkoski, Aleksandar
%D 2016
%T Evaluation of Intrusion Detection Systems in Virtualized Environments
%U https://opus.bibliothek.uni-wuerzburg.de/frontdoor/index/index/docId/14184

{Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments}. Milenkoski, Aleksandar; Jayaram, K. R.; Antunes, Nuno; Vieira, Marco; Kounev, Samuel; in {Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016)} (2016). IEEE Computer Society, {Washington DC, USA}.

{Acceptance rate (Full Paper): 45/130 = 34\%}

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.

@inproceedings{MiJaAnViKo2016-ISSRE-Quantifying,
  abstract = {With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.},
  address = {{Washington DC, USA}},
  author = {Milenkoski, Aleksandar and Jayaram, K. R. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel},
  booktitle = {{Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016)}},
  keywords = {Virtualization},
  month = 10,
  note = {{Acceptance rate (Full Paper): 45/130 = 34\%}},
  organization = {IEEE},
  publisher = {IEEE Computer Society},
  title = {{Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments}},
  year = 2016
}

%0 Conference Paper
%1 MiJaAnViKo2016-ISSRE-Quantifying
%A Milenkoski, Aleksandar
%A Jayaram, K. R.
%A Antunes, Nuno
%A Vieira, Marco
%A Kounev, Samuel
%B {Proceedings of The 27th IEEE International Symposium on Software Reliability Engineering (ISSRE 2016)}
%C {Washington DC, USA}
%D 2016
%I IEEE Computer Society
%T {Quantifying the Attack Detection Accuracy of Intrusion Detection Systems in Virtualized Environments}
%U http://ieeexplore.ieee.org/document/7774527/?reload=true
%X With the widespread adoption of virtualization, intrusion detection systems (IDSes) are increasingly being deployed in virtualized environments. When securing an environment, IT security officers are often faced with the question of how accurate deployed IDSes are at detecting attacks. To this end, metrics for assessing the attack detection accuracy of IDSes have been developed. However, these metrics are defined with respect to a fixed set of hardware resources available to the tested IDS. Therefore, IDSes deployed in virtualized environments featuring elasticity (i.e., on-demand allocation or deallocation of virtualized hardware resources during system operation) cannot be evaluated in an accurate manner using existing metrics. In this paper, we demonstrate the impact of elasticity on IDS attack detection accuracy. In addition, we propose a novel metric and measurement methodology for accurately quantifying the accuracy of IDSes deployed in virtualized environments featuring elasticity. We demonstrate their practical use through case studies involving commonly used IDSes.

{Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices}. Milenkoski, Aleksandar; Vieira, Marco; Kounev, Samuel; Avritzer, Alberto; Payne, Bryan D.; in {ACM Computing Surveys} (2015). 48(1) 12:1–12:41. ACM, New York, NY, USA.

{5-year Impact Factor (2014): 5.949}

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

{The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.}

@article{MiViKoAvPa2015-CSUR-IDSEval,
  abstract = {{The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.}},
  address = {New York, NY, USA},
  author = {Milenkoski, Aleksandar and Vieira, Marco and Kounev, Samuel and Avritzer, Alberto and Payne, Bryan D.},
  journal = {{ACM Computing Surveys}},
  keywords = {Reliability},
  month = {{September}},
  note = {{<b>5-year Impact Factor (2014): 5.949</b>}},
  number = 1,
  pages = {12:1–12:41},
  publisher = {ACM},
  title = {{Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices}},
  volume = 48,
  year = 2015
}

%0 Journal Article
%1 MiViKoAvPa2015-CSUR-IDSEval
%A Milenkoski, Aleksandar
%A Vieira, Marco
%A Kounev, Samuel
%A Avritzer, Alberto
%A Payne, Bryan D.
%C New York, NY, USA
%D 2015
%I ACM
%J {ACM Computing Surveys}
%N 1
%P 12:1--12:41
%T {Evaluating Computer Intrusion Detection Systems: A Survey of Common Practices}
%U http://dl.acm.org/authorize?N06203
%V 48
%X {The evaluation of computer intrusion detection systems (which we refer to as intrusion detection systems) is an active research area. In this paper, we survey and systematize common practices in the area of evaluation of intrusion detection systems. For this purpose, we define a design space structured into three parts: workload, metrics, and measurement methodology. We then provide an overview of the common practices in evaluation of intrusion detection systems by surveying evaluation approaches and methods related to each part of the design space. Finally, we discuss open issues and challenges focusing on evaluation methodologies for novel intrusion detection systems.}

{Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection}. Milenkoski, Aleksandar; Payne, Bryan D.; Antunes, Nuno; Vieira, Marco; Kounev, Samuel; Avritzer, Alberto; Luft, Matthias; in The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015) (2015). {Springer}.

{Acceptance Rate: 23%.}

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

{The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.}

@inproceedings{MiPaAnViKoAvLu2015-RAID-Challenges,
  abstract = {{The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.}},
  author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel and Avritzer, Alberto and Luft, Matthias},
  booktitle = {The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015)},
  keywords = {Virtualization},
  month = {{November}},
  note = {{Acceptance Rate: 23%.}},
  publisher = {{Springer}},
  title = {{Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection}},
  year = 2015
}

%0 Conference Paper
%1 MiPaAnViKoAvLu2015-RAID-Challenges
%A Milenkoski, Aleksandar
%A Payne, Bryan D.
%A Antunes, Nuno
%A Vieira, Marco
%A Kounev, Samuel
%A Avritzer, Alberto
%A Luft, Matthias
%B The 18th International Symposium on Research in Attacks, Intrusions, and Defenses (RAID 2015)
%D 2015
%I {Springer}
%T {Evaluation of Intrusion Detection Systems in Virtualized Environments Using Attack Injection}
%U http://link.springer.com/chapter/10.1007/978-3-319-26362-5_22
%X {The evaluation of intrusion detection systems (IDSes) is an active research area with many open challenges, one of which is the generation of representative workloads that contain attacks. In this paper, we propose a novel approach for the rigorous evaluation of IDSes in virtualized environments, with a focus on IDSes designed to detect attacks leveraging or targeting the hypervisor via its hypercall interface. We present hInjector, a tool for generating IDS evaluation workloads by injecting such attacks during regular operation of a virtualized environment. We demonstrate the application of our approach and show its practical usefulness by evaluating a representative IDS designed to operate in virtualized environments. The virtualized environment of the industry-standard benchmark SPECvirt_sc2013 is used as a testbed, whose drivers generate workloads representative of workloads seen in production environments. This work enables for the first time the injection of attacks in virtualized environments for the purpose of generating representative IDS evaluation workloads.}

{Challenges of Assessing the Hypercall Interface Robustness}. Carvalho, Diogo; Antunes, Nuno; Vieira, Marco; Milenkoski, Aleksandar; Kounev, Samuel; in The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015) (2015). {IEEE}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.

@inproceedings{CaAnViMiKo2015-DSN-Challenges,
  abstract = {Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.},
  author = {Carvalho, Diogo and Antunes, Nuno and Vieira, Marco and Milenkoski, Aleksandar and Kounev, Samuel},
  booktitle = {The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015)},
  keywords = {Reliability},
  month = {06},
  publisher = {{IEEE}},
  title = {{Challenges of Assessing the Hypercall Interface Robustness}},
  year = 2015
}

%0 Conference Paper
%1 CaAnViMiKo2015-DSN-Challenges
%A Carvalho, Diogo
%A Antunes, Nuno
%A Vieira, Marco
%A Milenkoski, Aleksandar
%A Kounev, Samuel
%B The 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2015)
%D 2015
%I {IEEE}
%T {Challenges of Assessing the Hypercall Interface Robustness}
%U http://2015.dsn.org/program/
%X Assessing the robustness of hypercall interfaces is essential to understand the dependability of virtualized infrastructures. However, the particularities in the domain raise particular challenges, as discussed in the present paper.

{Experience Report: An Analysis of Hypercall Handler Vulnerabilities}. Milenkoski, Aleksandar; Payne, Bryan D.; Antunes, Nuno; Vieira, Marco; Kounev, Samuel; in {Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) --- Research Track} (2014). IEEE Computer Society, {Washington DC, USA}.

Acceptance Rate: 25\%, Best Paper Award Nomination

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

{Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.}

@inproceedings{MiPaAnViKo2014-ISSRE-AnAnalHypHanVulns,
  abstract = {{Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.}},
  address = {{Washington DC, USA}},
  author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel},
  booktitle = {{Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) — Research Track}},
  keywords = {Reliability},
  month = 11,
  note = {Acceptance Rate: 25\%, <b>Best Paper Award Nomination</b>},
  organization = {IEEE},
  publisher = {IEEE Computer Society},
  title = {{Experience Report: An Analysis of Hypercall Handler Vulnerabilities}},
  year = 2014
}

%0 Conference Paper
%1 MiPaAnViKo2014-ISSRE-AnAnalHypHanVulns
%A Milenkoski, Aleksandar
%A Payne, Bryan D.
%A Antunes, Nuno
%A Vieira, Marco
%A Kounev, Samuel
%B {Proceedings of The 25th IEEE International Symposium on Software Reliability Engineering (ISSRE 2014) --- Research Track}
%C {Washington DC, USA}
%D 2014
%I IEEE Computer Society
%T {Experience Report: An Analysis of Hypercall Handler Vulnerabilities}
%U http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6982618
%X {Hypervisors are becoming increasingly ubiquitous with the growing proliferation of virtualized data centers. As a result, attackers are exploring vectors to attack hypervisors, against which an attack may be executed via several attack vectors such as device drivers, virtual machine exit events, or hypercalls. Hypercalls enable intrusions in hypervisors through their hypercall interfaces. Despite the importance, there is very limited publicly available information on vulnerabilities of hypercall handlers and attacks triggering them, which significantly hinders advances towards monitoring and securing these interfaces. In this paper, we characterize the hypercall attack surface based on analyzing a set of vulnerabilities of hypercall handlers. We systematize and discuss the errors that caused the considered vulnerabilities, and activities for executing attacks triggering them. We also demonstrate attacks triggering the considered vulnerabilities and analyze their effects. Finally, we suggest an action plan for improving the security of hypercall interfaces.}

{Technical Information on Vulnerabilities of Hypercall Handlers} Milenkoski, Aleksandar; Vieira, Marco; Payne, Bryan D.; Antunes, Nuno; Kounev, Samuel; (2014). SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC), {7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]

{Modern virtualized service infrastructures expose attack vectors that enable attacks of high severity, such as attacks targeting hypervisors. A malicious user of a guest VM (virtual machine) may execute an attack against the underlying hypervisor via hypercalls, which are software traps from a kernel of a fully or partially paravirtualized guest VM to the hypervisor. The exploitation of a vulnerability of a hypercall handler may have severe consequences such as altering hypervisor's memory, which may result in the execution of malicious code with hypervisor privilege. Despite the importance of vulnerabilities of hypercall handlers, there is not much publicly available information on them. This significantly hinders advances towards securing hypercall interfaces. In this work, we provide in-depth technical information on publicly disclosed vulnerabilities of hypercall handlers. Our vulnerability analysis is based on reverse engineering the released patches fixing the considered vulnerabilities. For each analyzed vulnerability, we provide background information essential for understanding the vulnerability, and information on the vulnerable hypercall handler and the error causing the vulnerability. We also show how the vulnerability can be triggered and discuss the state of the targeted hypervisor after the vulnerability has been triggered.}

@techreport{MiViPaAnKo2014-TechReport-TechInformation,
  abstract = {{Modern virtualized service infrastructures expose attack vectors that enable attacks of high severity, such as attacks targeting hypervisors. A malicious user of a guest VM (virtual machine) may execute an attack against the underlying hypervisor via hypercalls, which are software traps from a kernel of a fully or partially paravirtualized guest VM to the hypervisor. The exploitation of a vulnerability of a hypercall handler may have severe consequences such as altering hypervisor's memory, which may result in the execution of malicious code with hypervisor privilege. Despite the importance of vulnerabilities of hypercall handlers, there is not much publicly available information on them. This significantly hinders advances towards securing hypercall interfaces. In this work, we provide in-depth technical information on publicly disclosed vulnerabilities of hypercall handlers. Our vulnerability analysis is based on reverse engineering the released patches fixing the considered vulnerabilities. For each analyzed vulnerability, we provide background information essential for understanding the vulnerability, and information on the vulnerable hypercall handler and the error causing the vulnerability. We also show how the vulnerability can be triggered and discuss the state of the targeted hypervisor after the vulnerability has been triggered.}},
  address = {{7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}},
  author = {Milenkoski, Aleksandar and Vieira, Marco and Payne, Bryan D. and Antunes, Nuno and Kounev, Samuel},
  institution = {SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC)},
  keywords = {Reliability},
  month = {08},
  title = {{Technical Information on Vulnerabilities of Hypercall Handlers}},
  type = {{Technical Report SPEC-RG-2014-001 v.1.0}},
  year = 2014
}

%0 Report
%1 MiViPaAnKo2014-TechReport-TechInformation
%A Milenkoski, Aleksandar
%A Vieira, Marco
%A Payne, Bryan D.
%A Antunes, Nuno
%A Kounev, Samuel
%C {7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}
%D 2014
%T {Technical Information on Vulnerabilities of Hypercall Handlers}
%X {Modern virtualized service infrastructures expose attack vectors that enable attacks of high severity, such as attacks targeting hypervisors. A malicious user of a guest VM (virtual machine) may execute an attack against the underlying hypervisor via hypercalls, which are software traps from a kernel of a fully or partially paravirtualized guest VM to the hypervisor. The exploitation of a vulnerability of a hypercall handler may have severe consequences such as altering hypervisor's memory, which may result in the execution of malicious code with hypervisor privilege. Despite the importance of vulnerabilities of hypercall handlers, there is not much publicly available information on them. This significantly hinders advances towards securing hypercall interfaces. In this work, we provide in-depth technical information on publicly disclosed vulnerabilities of hypercall handlers. Our vulnerability analysis is based on reverse engineering the released patches fixing the considered vulnerabilities. For each analyzed vulnerability, we provide background information essential for understanding the vulnerability, and information on the vulnerable hypercall handler and the error causing the vulnerability. We also show how the vulnerability can be triggered and discuss the state of the targeted hypervisor after the vulnerability has been triggered.}

{On Benchmarking Intrusion Detection Systems in Virtualized Environments} Milenkoski, Aleksandar; Kounev, Samuel; Avritzer, Alberto; Antunes, Nuno; Vieira, Marco; (2013). SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC), {7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]

{Modern intrusion detection systems (IDSes) for virtualized environments are deployed in the virtualization layer with components inside the virtual machine monitor (VMM) and the trusted host virtual machine (VM). Such IDSes can monitor at the same time the network and host activities of all guest VMs running on top of a VMM being isolated from malicious users of these VMs. We refer to IDSes for virtualized environments as VMM-based IDSes. In this work, we analyze state-of-the-art intrusion detection techniques applied in virtualized environments and architectures of VMM-based IDSes. Further, we identify challenges that apply specifically to benchmarking VMM-based IDSes focussing on workloads and metrics. For example, we discuss the challenge of defining representative baseline benign workload profiles as well as the challenge of defining malicious workloads containing attacks targeted at the VMM. We also discuss the impact of on-demand resource provisioning features of virtualized environments (e.g., CPU and memory hotplugging, memory ballooning) on IDS benchmarking measures such as capacity and attack detection accuracy. Finally, we outline future research directions in the area of benchmarking VMM-based IDSes and of intrusion detection in virtualized environments in general.}

@techreport{MiKoAvAnVi2013-TechReport-OnBenchmarkingIDSes,
  abstract = {{Modern intrusion detection systems (IDSes) for virtualized environments are deployed in the virtualization layer with components inside the virtual machine monitor (VMM) and the trusted host virtual machine (VM). Such IDSes can monitor at the same time the network and host activities of all guest VMs running on top of a VMM being isolated from malicious users of these VMs. We refer to IDSes for virtualized environments as VMM-based IDSes. In this work, we analyze state-of-the-art intrusion detection techniques applied in virtualized environments and architectures of VMM-based IDSes. Further, we identify challenges that apply specifically to benchmarking VMM-based IDSes focussing on workloads and metrics. For example, we discuss the challenge of defining representative baseline benign workload profiles as well as the challenge of defining malicious workloads containing attacks targeted at the VMM. We also discuss the impact of on-demand resource provisioning features of virtualized environments (e.g., CPU and memory hotplugging, memory ballooning) on IDS benchmarking measures such as capacity and attack detection accuracy. Finally, we outline future research directions in the area of benchmarking VMM-based IDSes and of intrusion detection in virtualized environments in general.}},
  address = {{7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}},
  author = {Milenkoski, Aleksandar and Kounev, Samuel and Avritzer, Alberto and Antunes, Nuno and Vieira, Marco},
  institution = {SPEC Research Group - IDS Benchmarking Working Group, Standard Performance Evaluation Corporation (SPEC)},
  keywords = {Reliability},
  month = {06},
  title = {{On Benchmarking Intrusion Detection Systems in Virtualized Environments}},
  type = {{Technical Report SPEC-RG-2013-002 v.1.0}},
  year = 2013
}

%0 Report
%1 MiKoAvAnVi2013-TechReport-OnBenchmarkingIDSes
%A Milenkoski, Aleksandar
%A Kounev, Samuel
%A Avritzer, Alberto
%A Antunes, Nuno
%A Vieira, Marco
%C {7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}
%D 2013
%T {On Benchmarking Intrusion Detection Systems in Virtualized Environments}
%X {Modern intrusion detection systems (IDSes) for virtualized environments are deployed in the virtualization layer with components inside the virtual machine monitor (VMM) and the trusted host virtual machine (VM). Such IDSes can monitor at the same time the network and host activities of all guest VMs running on top of a VMM being isolated from malicious users of these VMs. We refer to IDSes for virtualized environments as VMM-based IDSes. In this work, we analyze state-of-the-art intrusion detection techniques applied in virtualized environments and architectures of VMM-based IDSes. Further, we identify challenges that apply specifically to benchmarking VMM-based IDSes focussing on workloads and metrics. For example, we discuss the challenge of defining representative baseline benign workload profiles as well as the challenge of defining malicious workloads containing attacks targeted at the VMM. We also discuss the impact of on-demand resource provisioning features of virtualized environments (e.g., CPU and memory hotplugging, memory ballooning) on IDS benchmarking measures such as capacity and attack detection accuracy. Finally, we outline future research directions in the area of benchmarking VMM-based IDSes and of intrusion detection in virtualized environments in general.}

{Cloud Usage Patterns: A Formalism for Description of Cloud Usage Scenarios} Milenkoski, Aleksandar; Iosup, Alexandru; Kounev, Samuel; Sachs, Kai; Rygielski, Piotr; Ding, Jason; Cirne, Walfredo; Rosenberg, Florian; (2013). SPEC Research Group - Cloud Working Group, Standard Performance Evaluation Corporation (SPEC), {7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

{Cloud computing is becoming an increasingly lucrative branch of the existing information and communication technologies (ICT). Enabling a debate about cloud usage scenarios can help with attracting new customers, sharing best-practices, and designing new cloud services. In contrast to previous approaches, which have attempted mainly to formalize the common service delivery models (i.e., Infrastructure-as-a-Service, Platform-as-a-Service, and Software-as-a-Service), in this work, we propose a formalism for describing common cloud usage scenarios referred to as cloud usage patterns. Our formalism takes a structuralist approach allowing decomposition of a cloud usage scenario into elements corresponding to the common cloud service delivery models. Furthermore, our formalism considers several cloud usage patterns that have recently emerged, such as hybrid services and value chains in which mediators are involved, also referred to as value chains with mediators. We propose a simple yet expressive textual and visual language for our formalism, and we show how it can be used in practice for describing a variety of real-world cloud usage scenarios. The scenarios for which we demonstrate our formalism include resource provisioning of global providers of infrastructure and/or platform resources, online social networking services, user-data processing services, online customer and ticketing services, online asset management and banking applications, CRM (Customer Relationship Management) applications, and online social gaming applications.}

@techreport{MiIoKoSaRyDiCiRo2013-TechReport-CloudUsagePatterns,
  abstract = {{Cloud computing is becoming an increasingly lucrative branch of the existing information and communication technologies (ICT). Enabling a debate about cloud usage scenarios can help with attracting new customers, sharing best-practices, and designing new cloud services. In contrast to previous approaches, which have attempted mainly to formalize the common service delivery models (i.e., Infrastructure-as-a-Service, Platform-as-a-Service, and Software-as-a-Service), in this work, we propose a formalism for describing common cloud usage scenarios referred to as cloud usage patterns. Our formalism takes a structuralist approach allowing decomposition of a cloud usage scenario into elements corresponding to the common cloud service delivery models. Furthermore, our formalism considers several cloud usage patterns that have recently emerged, such as hybrid services and value chains in which mediators are involved, also referred to as value chains with mediators. We propose a simple yet expressive textual and visual language for our formalism, and we show how it can be used in practice for describing a variety of real-world cloud usage scenarios. The scenarios for which we demonstrate our formalism include resource provisioning of global providers of infrastructure and/or platform resources, online social networking services, user-data processing services, online customer and ticketing services, online asset management and banking applications, CRM (Customer Relationship Management) applications, and online social gaming applications.}},
  address = {{7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}},
  author = {Milenkoski, Aleksandar and Iosup, Alexandru and Kounev, Samuel and Sachs, Kai and Rygielski, Piotr and Ding, Jason and Cirne, Walfredo and Rosenberg, Florian},
  institution = {SPEC Research Group - Cloud Working Group, Standard Performance Evaluation Corporation (SPEC)},
  keywords = {Virtualization},
  month = {05},
  title = {{Cloud Usage Patterns: A Formalism for Description of Cloud Usage Scenarios}},
  type = {{Technical Report SPEC-RG-2013-001 v.1.0.1}},
  year = 2013
}

%0 Report
%1 MiIoKoSaRyDiCiRo2013-TechReport-CloudUsagePatterns
%A Milenkoski, Aleksandar
%A Iosup, Alexandru
%A Kounev, Samuel
%A Sachs, Kai
%A Rygielski, Piotr
%A Ding, Jason
%A Cirne, Walfredo
%A Rosenberg, Florian
%C {7001 Heritage Village Plaza Suite 225, Gainesville, VA 20155, USA}
%D 2013
%T {Cloud Usage Patterns: A Formalism for Description of Cloud Usage Scenarios}
%U http://research.spec.org/fileadmin/user_upload/documents/rg_cloud/endorsed_publications/SPEC-RG-2013-001_CloudUsagePatterns.pdf
%X {Cloud computing is becoming an increasingly lucrative branch of the existing information and communication technologies (ICT). Enabling a debate about cloud usage scenarios can help with attracting new customers, sharing best-practices, and designing new cloud services. In contrast to previous approaches, which have attempted mainly to formalize the common service delivery models (i.e., Infrastructure-as-a-Service, Platform-as-a-Service, and Software-as-a-Service), in this work, we propose a formalism for describing common cloud usage scenarios referred to as cloud usage patterns. Our formalism takes a structuralist approach allowing decomposition of a cloud usage scenario into elements corresponding to the common cloud service delivery models. Furthermore, our formalism considers several cloud usage patterns that have recently emerged, such as hybrid services and value chains in which mediators are involved, also referred to as value chains with mediators. We propose a simple yet expressive textual and visual language for our formalism, and we show how it can be used in practice for describing a variety of real-world cloud usage scenarios. The scenarios for which we demonstrate our formalism include resource provisioning of global providers of infrastructure and/or platform resources, online social networking services, user-data processing services, online customer and ticketing services, online asset management and banking applications, CRM (Customer Relationship Management) applications, and online social gaming applications.}

{HInjector: Injecting Hypercall Attacks for Evaluating VMI-based Intrusion Detection Systems}. Milenkoski, Aleksandar; Payne, Bryan D.; Antunes, Nuno; Vieira, Marco; Kounev, Samuel; in The 2013 Annual Computer Security Applications Conference (ACSAC 2013) (2013). {Applied Computer Security Associates (ACSA)}, Maryland, USA.

[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

@inproceedings{MiPaAnViKo2013-ACSAC-HInjector,
  address = {Maryland, USA},
  author = {Milenkoski, Aleksandar and Payne, Bryan D. and Antunes, Nuno and Vieira, Marco and Kounev, Samuel},
  booktitle = {The 2013 Annual Computer Security Applications Conference (ACSAC 2013)},
  keywords = {Reliability},
  month = {03},
  publisher = {{Applied Computer Security Associates (ACSA)}},
  title = {{HInjector: Injecting Hypercall Attacks for Evaluating VMI-based Intrusion Detection Systems}},
  year = 2013
}

%0 Conference Paper
%1 MiPaAnViKo2013-ACSAC-HInjector
%A Milenkoski, Aleksandar
%A Payne, Bryan D.
%A Antunes, Nuno
%A Vieira, Marco
%A Kounev, Samuel
%B The 2013 Annual Computer Security Applications Conference (ACSAC 2013)
%C Maryland, USA
%D 2013
%I {Applied Computer Security Associates (ACSA)}
%T {HInjector: Injecting Hypercall Attacks for Evaluating VMI-based Intrusion Detection Systems}
%U https://www.acsac.org/2013/program-files/p65.html

{Towards Benchmarking Intrusion Detection Systems for Virtualized Cloud Environments}. Milenkoski, Aleksandar; Kounev, Samuel; in Proceedings of the 7th International Conference for Internet Technology and Secured Transactions (ICITST 2012) (2012). 562–563. IEEE, New York, USA.

[ Abstract ]
[ BibTeX ]
[ EndNote ]
[ URL ]
[ Download ]

{Many recent research works propose novel architectures of intrusion detection systems specifically designed to operate in virtualized environments. However, little attention has been given to the evaluation and benchmarking of such architectures with respect to their performance and dependability. In this paper, we present a research roadmap towards developing a framework for benchmarking intrusion detection systems for cloud environments in a scientifically rigorous and a representative manner.}

@inproceedings{MiKo2011-ICITST-TowardsBenchmarking,
  abstract = {{Many recent research works propose novel architectures of intrusion detection systems specifically designed to operate in virtualized environments. However, little attention has been given to the evaluation and benchmarking of such architectures with respect to their performance and dependability. In this paper, we present a research roadmap towards developing a framework for benchmarking intrusion detection systems for cloud environments in a scientifically rigorous and a representative manner.}},
  address = {New York, USA},
  author = {Milenkoski, Aleksandar and Kounev, Samuel},
  booktitle = {Proceedings of the 7th International Conference for Internet Technology and Secured Transactions (ICITST 2012)},
  keywords = {Virtualization},
  month = 12,
  pages = {562–563},
  publisher = {IEEE},
  title = {{Towards Benchmarking Intrusion Detection Systems for Virtualized Cloud Environments}},
  year = 2012
}

%0 Conference Paper
%1 MiKo2011-ICITST-TowardsBenchmarking
%A Milenkoski, Aleksandar
%A Kounev, Samuel
%B Proceedings of the 7th International Conference for Internet Technology and Secured Transactions (ICITST 2012)
%C New York, USA
%D 2012
%I IEEE
%P 562--563
%T {Towards Benchmarking Intrusion Detection Systems for Virtualized Cloud Environments}
%U http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6470873
%X {Many recent research works propose novel architectures of intrusion detection systems specifically designed to operate in virtualized environments. However, little attention has been given to the evaluation and benchmarking of such architectures with respect to their performance and dependability. In this paper, we present a research roadmap towards developing a framework for benchmarking intrusion detection systems for cloud environments in a scientifically rigorous and a representative manner.}

Publications of Aleksandar Milenkoski

Data privacy protection

Data privacy protection

Picture credits