ComBench: A Benchmarking Framework for Publish/Subscribe Communication Protocols Under Network Limitations. Herrnleben, Stefan; Leidinger, Maximilian; Lesch, Veronika; Prantl, Thomas; Grohmann, Johannes; Krupitzer, Christian; Kounev, Samuel; in Performance Evaluation Methodologies and Tools, Q. Zhao, L. Xia (Hrsg.) (2021). 72–92.
Efficient and dependable communication is a highly relevant aspect for Internet of Things (IoT) systems in which tiny sensors, actuators, wearables, or other smart devices exchange messages. Various publish/subscribe protocols address the challenges of communication in IoT systems. The selection process of a suitable protocol should consider the communication behavior of the application, the protocol's performance, the resource requirements on the end device, and the network connection quality, as IoT environments often rely on wireless networks. Benchmarking is a common approach to evaluate and compare systems, considering the performance and aspects like dependability or security. In this paper, we present our IoT communication benchmarking framework ComBench for publish/subscribe protocols focusing on constrained networks with varying quality conditions. The benchmarking framework supports system designers, software engineers, and application developers to select and investigate the behavior of communication protocols. Our benchmarking framework contributes to (i) show the impact of fluctuating network quality on communication, (ii) compare multiple protocols, protocol features, and protocol implementations, and (iii) analyze scalability, robustness, and dependability of clients, networks, and brokers in different scenarios. Our case study demonstrates the applicability of our framework to support the decision for the best-suited protocol in various scenarios.
A Simulation-based Optimization Framework for Online Adaptation of Networks. Herrnleben, Stefan; Grohmann, Johannes; Rygielski, Pitor; Lesch, Veronika; Krupitzer, Christian; Kounev, Samuel; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools), H. Song, D. Jiang (Hrsg.) (2021). 513–532.
Today's data centers face a rapid change of deployed services, growing complexity, and increasing performance requirements. Customers expect not only round-the-clock availability of the hosted services but also high responsiveness. Besides optimizing software architectures and deployments, networks have to be adapted to handle the changing and volatile demands. Approaches from self-adaptive systems can be used for optimizing data center networks to continuously meet Service Level Agreements (SLAs) between data center operators and customers. However, existing approaches focus only on specific objectives like topology design, power optimization, or traffic engineering. In this paper, we present an extensible framework that analyzes networks using different types of simulation and adapts them subject to multiple objectives using various adaptation techniques. Analyzing each suggested adaptation ensures that performance requirements and SLAs are continuously met. We evaluate our framework w.r.t. (i) general requirements and assessments of languages and frameworks for adaptation models, (ii) finding Pareto-optimal solutions considering a multi-dimensional cost model, and (iii) scalability. The evaluation shows that our approach detects the bottlenecks and the violated SLAs correctly, outputs valid and cost-optimal adaptations, and keeps the runtime for the adaptation process constant even with increasing network size and an increasing number of alternative configurations.
The Power of Composition: Abstracting a Multi-Device SDN Data Path Through a Single API. Geissler, Stefan; Herrnleben, Stefan; Bauer, Robert; Grigorjew, Alexej; Zinner, Thomas; Jarschel, Michael; in IEEE Trans. Netw. Serv. Manag. (2020). 17(2) 722–735.
An IoT Network Emulator for Analyzing the Influence of Varying Network Quality. Herrnleben, Stefan; Ailabouni, Rudy; Grohmann, Johannes; Prantl, Thomas; Krupitzer, Christian; Kounev, Samuel; in Proceedings of the 12th EAI International Conference on Simulation Tools and Techniques (SIMUtools) (2020).
IoT devices often communicate over wireless or cellular networks with varying connection quality. These fluctuations are caused, among others, by the free-space path loss (FSPL), buildings, topological obstacles, weather, and mobility of the receiver. Varying signal quality affects bandwidth, transmission delays, packet loss, and jitter. Mobile IoT applications exposed to varying connection characteristics have to handle such variations and take them into account during development and testing. However, tests in real mobile networks are complex and challenging to reproduce. Therefore, network emulators can be used to simulate the behavior of real-world networks by adding artificial disturbance. However, existing network emulators often require a lot of technical knowledge and complex setup. Integrating such emulators into automated software testing pipelines could be a challenging task. In this paper, we propose a framework for emulating IoT networks with varying quality characteristics. An existing base emulator is used and integrated into our framework enabling the user to utilize it without extensive network expertise and configuration effort. The evaluation proves that our framework can simulate a variety of different network quality characteristics as well as emulating real-world network traces.
Model-based Performance Predictions for SDN-based Networks: A Case Study. Herrnleben, Stefan; Rygielski, Piotr; Grohmann, Johannes; Eismann, Simon; Hossfeld, Tobias; Kounev, Samuel; in Proceedings of the 20th International GI/ITG Conference on Measurement, Modelling and Evaluation of Computing Systems (2020).
Emerging paradigms for network virtualization like Software-Defined Networking (SDN) and Network Functions Virtualization (NFV) form new challenges for accurate performance modeling and analysis tools. Therefore, performance modeling and prediction approaches that support SDN or NFV technologies help system operators to analyze the performance of a data center and its corresponding network. The Descartes Network Infrastructures (DNI) offers a high-level descriptive language to model SDN-based networks, which can be transformed into various predictive modeling formalisms. However, these modeling concepts have not yet been evaluated in a realistic scenario. In this paper, we present an extensive case study evaluating the DNI modeling capabilities, the transformations to predictive models, and the performance prediction using the OMNeT++ and SimQPN simulation frameworks. We present five realistic scenarios of a content distribution network (CDN), compare the performance predictions with real-world measurements, and discuss modeling gaps and calibration issues causing mispredictions in some scenarios.
TableVisor 2.0: Towards Full-Featured, Scalable and Hardware-Independent Multi Table Processing. Geissler, Stefan; Gebert, Steffen; Herrnleben, Stefan; Zinner, Thomas; Bauer, Robert; Jarschel, Michael; in NetSoft 2017 (2017).
Best Student Paper Award
Modern Software Defined Networking (SDN) appli- cations rely on sophisticated packet processing. However, there is a mismatch between control plane requirements and data plane capabilities caused by increasing hardware heterogeneity. To overcome this challenge, we propose TableVisor, a proxy-layer for the OpenFlow control channel that enables the flexible and scalable abstraction of multiple physical devices into one emu- lated data plane switch that meets the requirements of the control plane application. TableVisor registers with the SDN controller as a single switch with use-case specific capabilities. It translates the instructions and rules from the control application towards the appropriate physical device where they are executed. In this paper, we present the updated architecture and functionality of TableVisor as well as first evaluation results based on testbed experiments.