edit · history · print

Accurate and real-time traffic measurement is becoming increasingly critical for large variety of applications including accounting, bandwidth provisioning and security analysis. Existing network measurement techniques, however, have major difficulty dealing with large number of flows in today's high-speed networks and offer limited scalability with increasing link speeds. Consequently, the current state of the art solutions have to resort to conservative sampling of the traffic stream and/or to account for only a few frequent flows that often fail to provide accurate estimates of traffic features.

Fundamentally, traffic measurement involves counting number of packets that satisfy some criteria, commonly referred to as user query or a rule, over a period of time. The traffic is measured in terms of flows, where a flow refers to a set of packets that have the same n-tuple value in their header fields. Typical definitions of flow include the 6-tuple:{prt, tos, sip, spt, dip, dpt} where, prt is the protocol field, tos is type of service, sip and dip are the source and destination IP addresses and spt and dpt are the source and destination ports, respectively.


Figure:1 Traditional per-flow measurement paradigm

Traditionally, measurement schemes have operated by maintaining unique per-flow based counters on high-density storage media, followed by aggregation of selected counters to answer queries. This mechanism is illustrated in Figure-1. An inherent and increasingly-widening performance gap between high-density storage access time and network bandwidth coupled with a significant I/O overhead makes it difficult to operate the paradigm in real-time. As a result, traditional schemes do not offer scalable measurement solutions for high data rate networks and have to resort to sampling and/or offline processing. Cisco's NetFlow is one such widely deployed sample-based traffic measurement solution.


Figure:2 Query driven measurement paradigm

The key issue with sample based solutions is measurement accuracy. We argue that a scalable solution for real-time and accurate measurement of traffic has to dispose of conventional per-flow based statistics collection. Instead, in this work, we propose a query-driven measurement methodology that works by profiling passing traffic according to the given query to collect information of interest in real-time as shown in Figure-2. Unlike existing techniques, our solution processes streaming traffic at link speeds and hence, does not compromise measurement accuracy due to sampling.

The focus of this work is in evaluating the query-driven measurement paradigm utilizing embedded systems. An initial prototype has already been developed on an FPGA platform. A high level diagram of the prototype is presented in Figure-3. The architecture consists of a hardware-software co-designed solution that contains a highly-parallel and scalable array of processing elements in hardware where the user queries can be dynamically and independently mapped. The mapping and interaction with the user is controlled through a control processor and glue logic interface. Further details of the prototype solution have recently been published in ANCS'08.


Figure:3 A Parallel and Pipelined Architecture for Programmable Real-time Measurements

Publications:

  1. Faisal Khan, Nicholas Hosein, Chen-Nee Chuah, Soheil Ghiasi, "Streaming Solutions For Fine-Grained Network Traffic Measurements And Analysis", ACM/IEEE Symposium on Architectures for Networking and Communication Systems (ANCS), pp. 227-238, October 2011
  2. Faisal Khan, Nicholas Hosein, Scott Vernon, Soheil Ghiasi, "BURAQ: A Dynamically Reconfigurable System for Stateful Measurement of Network Traffic", 18th IEEE Annual International Symposium on Field-Programmable Custom Computing Machines, pp.185-192, May 2010 (24/132: 18% acceptance rate)
  3. Faisal Khan, Lihua Yuan, Chen-Nee Chuah, Soheil Ghiasi, "A Programmable Architecture for Scalable and Real-time Network Traffic Measurements", In proceedings of the 4th ACM/IEEE Symposium on Architecture for networking and communications systems, ANCS'08. Slides

Presentations:

Industrial Affiliates Conference-2009, Jan. 16th '09, UC-Davis Poster

Projects:

We have several opportunities for students who would like to work in this area and/or play with FPGAs. We are looking for students who are already around the campus and have 'very' good digital designing/programming language background. Here are some potential projects

  • Development of PC front-end to automate query composition and FPGA placement (Skills Reqd: C/C++)
  • FPGA board (ML505) Integration with PC using RocketIO interface (Skills Reqd: C/C++, Verilog, Working knowledge of Xilinx EDK)
  • FPGA board (ML505) Integration with PC using Ethernet interface (Skills Reqd: C/C++, Verilog, Working knowledge of Xilinx EDK)
  • Test-suite and verification methodology development. (Skills Reqd: C/C++)
Page last modified on January 20, 2012, at 09:39 PM