Many networking applications require fast state lookups in a concurrent state machine, which tracks the state of a large number of flows simultaneously. We consider the question of how to compactly represent such concurrent state machines. To achieve compactness, we consider data structures for Approximate Concurrent State Machines (ACSMs) that can return false positives, false negatives, or a “don’t know” response. We describe three techniques based on Bloom filters and hashing, and evaluate them using both theoretical analysis and simulation. Our analysis leads us to an extremely efficient hashing-based scheme with several parameters that can be chosen to trade off space, computation, and the impact of errors. Our hashing approach also yields a simple alternative structure with the same functionality as a counting Bloom filter that uses much less space. We show how ACSMs can be used for video congestion control. Using an ACSM, a router can implement sophisticated Active Queue Management (AQM) techniques for video traffic (without the need for standards changes to mark packets or change video formats), with a factor of four reduction in memory compared to full-state schemes and with very little error. We also show that ACSMs show promise for real-time detection of P2P traffic.

We consider the problem of establishing jam-resistant, wireless, omnidirectional communication channels when there is no initial shared secret. No existing system achieves this. We propose a general algorithm for this problem, the BBC algorithm, and give several instantiations of it. We develop and analyze this algorithm within the framework of a new type of code, concurrent codes, which are those superimposed codes that allow efficient decoding. Finally, we propose the Universal Concurrent Code algorithm, and prove that it covers all possible concurrent codes, and give connections between its theory and that of monotone Boolean functions.

The sophistication and complexity of analysis performed by today’s network intrusion prevention systems (IPSs) benefits greatly from implementation using general-purpose CPUs. Yet the performance of such CPUs increasingly lags behind that necessary to process today’s high-rate traffic streams. A key observation, however, is that much of the traffic comprising a high-volume stream can, after some initial analysis, be qualified as “likely uninteresting.” To this end, we have developed an in-line, FPGA-based IPS accelerator, the Shunt, using the NetFPGA2 platform. The Shunt functions as the forwarding device used by the IPS; it alone processes the bulk of the traffic, offloading the memory bus and leaving the CPU free to inspect the subset of the traffic deemed germane for security analysis. To do so, the Shunt maintains several large state tables indexed by packet header fields, including IP/TCP flags, source and destination IP addresses, and connection tuples. The tables yield decision values the element makes on a packet-by-packet basis: forward the packet, drop it, or divert it through the IPS. By manipulating table entries, the IPS can specify the traffic it wishes to examine, directly block malicious traffic, and “cut through ” traffic streams once it has had an opportunity to “vet ” them, all on a fine-grained basis. We base our design on a novel series of caches, with a “fail safe ” miss policy, coupled to a host PC to handle both cache management and higher level IPS analysis. The design requires only 2 MB of SRAM for its extensive caches, and can support

While the problem of general packet classification has received a great deal of attention from researchers over the last ten years, there is still no really satisfactory solution. Ternary Content Addressable Memory (TCAM), although widely used in practice, is both expensive and consumes a lot of power. Algorithmic solutions, which rely on commodity memory chips, are relatively inexpensive and power-efficient, but have not been able to match the generality and performance of TCAMs. In this paper we propose a new approach to packet classification, which combines architectural and algorithmic techniques. Our starting point is the well-known crossproducting algorithm, which is fast but has significant memory overhead due to the extra rules needed to represent the crossproducts. We show how to modify the crossproduct method in a way that drastically reduces the memory required, without compromising on performance. We avoid unnecessary accesses to off-chip memory by filtering off-chip accesses using on-chip Bloom filters. For packets that match p rules in a rule set, our algorithm requires just 4 + p + ǫ independent memory accesses on average, to return all matching rules, where ǫ ≪ 1 is a small constant that depends on the false positive rate of the Bloom filters. Each memory access is just 256 bits, making it practical to classify small packets at OC-192 link rates using two commodity SRAM chips. For rule set sizes ranging from a few hundred to several thousand filters, the average rule set expansion factor attributable to the algorithm is just 1.2. The memory consumption per rule is 36 bytes in the average case. 1 1

Stateful, in-depth, inline traffic analysis for intrusion detection and prevention is growing increasingly more difficult as the data rates of modern networks rise. Yet it remains the case that in many environments, much of the traffic comprising a high-volume stream can, after some initial analysis, be qualified as of “likely uninteresting.” We present a combined hardware/software architecture, Shunting, that provides a lightweight mechanism for an intrusion prevention system (IPS) to take advantage of the “heavy-tailed ” nature of network traffic to offload work from software to hardware. The primary innovation of Shunting is the introduction of a simple in-line hardware element that caches rules for IP addresses and connection 5-tuples, as well as fixed rules for IP/TCP flags. The caches, using a highest-priority match, yield a per-packet decision: forward the packet; drop it; or divert it through the IPS. By manipulating cache entries, the IPS can specify what traffic it no longer wishes to examine, including directly blocking malicious sources or cutting through portions of a single flow once the it has had an opportunity to “vet ” them, all on a fine-grained basis. We have implemented a prototype Shunt hardware design using the NetFPGA 2 platform, capable of Gigabit Ethernet operation. In addition, we have adapted the Bro intrusion detection system to utilize the Shunt framework to offload less-interesting traffic. We evaluate the effectiveness of the resulting system using traces from three sites, finding that the IDS can use this mechanism to offload 55%–90 % of the traffic, as well as gaining intrusion prevention functionality.

Abstract — High-speed packet content inspection and filtering devices rely on a fast multi-pattern matching algorithm which is used to detect predefined keywords or signatures in the packets. Multi-pattern matching is known to require intensive memory accesses and is often a performance bottleneck. Hence specialized hardware-accelerated algorithms are required for line-speed packet processing. We present hardware-implementable pattern matching algorithm for content filtering applications, which is scalable in terms of speed, the number of patterns and the pattern length. Our algorithm is based on a memory efficient multi-hashing data structure called Bloom filter. We use embedded on-chip memory blocks in FPGA/VLSI chips to construct Bloom filters which can suppress a large fraction of memory accesses and speed up string matching. Based on this concept, we first present a simple algorithm which can scan for several thousand short (up to 16 bytes) patterns at multi-gigabit per second speeds with a moderately small amount of embedded memory and a few mega bytes of external memory. Furthermore, we modify this algorithm to be able to handle arbitrarily large strings at the cost of a little more on-chip memory. We demonstrate the merit of our algorithm through theoretical analysis and simulations performed on Snort’s string set.

In a multiple-choice hashing scheme, each item is stored in one of d> = 2 possible hash tablebuckets. The availability of these multiple choices allows for a substantial reduction in the maximum load of the buckets. However, a lookup may now require examining each of the d locations. Forapplications where this cost is undesirable, Song et al. propose keeping a summary that allows one to determine which of the d locations is appropriate for each item, where the summary may allowfalse positives for items not in hash table. We propose alternative, simple constructions of such summaries that use less space for both the summary and the underlying hash table. Moreover, ourconstructions are easily analyzable and tunable.

Distribution A, Approved for public release, distribution unlimited Abstract. We consider the problem of establishing jam-resistant, wireless, omnidirectional communication channels when there is no initial shared secret. No existing system achieves this. We propose a general algorithm for this problem, the BBC algorithm, and give several instantiations of it. We develop and analyze this algorithm within the framework of a new type of code, concurrent codes, which are those superimposed codes that allow efficient decoding. Finally, we propose the Universal Concurrent Code algorithm, and prove that it covers all possible concurrent codes, and give connections between its theory and that of monotone Boolean functions.

Abstract — Hash tables are a fundamental data structure in many network applications, including route lookups, packet classification and monitoring. Often a part of the data path, they need to operate at wire-speed. However, several associative memory accesses are needed to resolve collisions, making them slower than required. This motivates us to consider minimal perfect hashing schemes, which reduce the number of memory accesses to just 1 and are also space-efficient. Existing perfect hashing algorithms are not tailored for network applications because they take too long to construct and are hard to implement in hardware. This paper introduces a hardware-friendly scheme for minimal perfect hashing, with space requirement approaching 3.7 times the information theoretic lower bound. Our construction is several orders faster than existing perfect hashing schemes. Instead of using the traditional mapping-partitioning-searching methodology, our scheme employs a Bloom filter, which is known for its simplicity and speed. We extend our scheme to the dynamic setting, thus handling insertions and deletions. I.

Abstract Hashing is an extremely useful technique for a variety of high-speed packet-processing applications in routers. In this chapter, we survey much of the recent work in this area, paying particular attention to the interaction between theoretical and applied research. We assume very little background in either the theory or applications of hashing, reviewing the fundamentals as necessary. 1