CommTech Seminar: Network Monitoring
Scalable Streaming Analytics for Network Monitoring
Prof. Marco Canini
Assistant professor in Computer Science, KAUST
DEIB - Seminar Room
October 18th, 2016
2.00 pm
Prof. Marco Canini
Assistant professor in Computer Science, KAUST
DEIB - Seminar Room
October 18th, 2016
2.00 pm
Abstract
With traffic rates already at hundreds Gbps to Tbps ranges, current mechanisms for monitoring network traffic are ill-suited for real-time analysis of security problems or performance troubleshooting because they do not return traffic statistics at the appropriate level of granularity.
Programmable switches potentially make it easier to perform flexible network monitoring queries at line rate, and scalable stream processing systems make it possible to fuse data streams to answer more sophisticated queries about the network in real-time. Processing such queries at high traffic rates, however, requires both the switches and the query processing system to iteratively filter the traffic.
This talk presents the design, implementation, and evaluation of Sonata, a stream-based network monitoring system that allows an operator to express network-wide queries that return only the traffic pertaining to the query, with as little extraneous traffic as possible and at the appropriate granularity. Given a query, Sonata automatically determines how to partition the query across the network switches and the stream processing system, performing as much filtering as possible in the switches and iteratively refining these filters to ensure that the traffic that the stream processor sees contains as little extraneous traffic as possible. We implement several example queries that are motivated by real-world security and troubleshooting scenarios and evaluate them using traffic traces from one of the world's largest Internet exchange points. Our evaluation shows that Sonata can reduce the data rate at the stream processor by four orders of magnitude and the number of counters by five orders of magnitude, relative to state of the art stream processing systems.
Programmable switches potentially make it easier to perform flexible network monitoring queries at line rate, and scalable stream processing systems make it possible to fuse data streams to answer more sophisticated queries about the network in real-time. Processing such queries at high traffic rates, however, requires both the switches and the query processing system to iteratively filter the traffic.
This talk presents the design, implementation, and evaluation of Sonata, a stream-based network monitoring system that allows an operator to express network-wide queries that return only the traffic pertaining to the query, with as little extraneous traffic as possible and at the appropriate granularity. Given a query, Sonata automatically determines how to partition the query across the network switches and the stream processing system, performing as much filtering as possible in the switches and iteratively refining these filters to ensure that the traffic that the stream processor sees contains as little extraneous traffic as possible. We implement several example queries that are motivated by real-world security and troubleshooting scenarios and evaluate them using traffic traces from one of the world's largest Internet exchange points. Our evaluation shows that Sonata can reduce the data rate at the stream processor by four orders of magnitude and the number of counters by five orders of magnitude, relative to state of the art stream processing systems.
Short Bio
Marco Canini is an assistant professor in Computer Science at KAUST. Marco obtained his Ph.D. in computer science and engineering from the University of Genoa in 2009 after spending the last year as a visiting student at the University of Cambridge, Computer Laboratory. He was a postdoctoral researcher at EPFL from 2009 to 2012 and after that a senior research scientist for one year at Deutsche Telekom Innovation Labs & TU Berlin. Before joining KAUST, he was an assistant professor at the Université catholique de Louvain. He also held positions at Intel Research and Google.