In a multiple-choice hashing scheme, each item is stored in one of P possible hash table buckets. The availability of these multiple choices allows for a substantial reduction in the maximum load of the buckets. However, a lookup may now require examining each of the locations. For applications where this cost is undesirable, Song et al. propose keeping a summary that allows one to determine which of the locations is appropriate for each item, where the summary may allow false positives for items not in hash table. We propose alternative, simple constructions of such summaries that use less space for both the summary and the underlying hash table. Moreover, our constructions are easily analyzable and tunable.

Packet classification continues to be an important challenge in network processing. It requires matching each packet against a database of rules and forwarding the packet according to the highest priority matching rule. Within the packet classification hash-based algorithms, an algorithm that is gaining interest is the tuple space search algorithm that groups the rules into a set of tuple spaces according to their prefix lengths. An incoming packet can now be matched to the rules in a group by taking into consideration only those prefixes specified by the tuples. More importantly, matching of an incoming packet can now be performed in parallel over all tuples. Within these tuple spaces, a drawback of utilizing hashing is that certain rules will be mapped to the same location, also called collision. The negative effect of such collision is that it will result in multiple memory accesses and subsequently longer processing time. In this paper, we propose to use a pruned counting Bloom filter to reduce collisions in the tuple space packet classification algorithm. This approach decreases the number of collisions and memory accesses in the rule set hash table in comparison to a traditional hashing system. We propose to utilize the pruned counting Bloom filter to decrease the number of collisions. More specifically, we investigate several well-known hashing functions and determine the number of collisions and show that utilizing the pruned counting Bloom filter the number of collisions can be further reduced by at least 4 % and by at most 32% for real rule sets.

Challenges in addressing the memory bottleneck have made it difficult to design a packet processing platform that simultaneously achieves both ease-of-programming and high performance. Today’s commercial processors support two architectural mechanisms–namely, hardware multithreading and caching–to overcome the memory bottleneck. The configurations of these mechanisms (e.g., cache capacity, number of threads per processor core) are fixed at processordesign time. The relative effectiveness of these mechanisms, however, varies significantly with application, traffic, and system characteristics. Thus, programmers often struggle to achieve high performance from a processor that is not well-suited to a particular deployment. To address this challenge, we first make a case for, and then develop a malleable processor architecture that facilitates the dynamic reconfiguration of cache capacity and number of threads to best-suit the needs of each deployment. We then present an algorithm that can determine the optimal thread-cache balance at run-time. The combination of these two allows us to simultaneously achieve the goals of ease-of-programming and high performance. We demonstrate that our processor outperforms a processor similar to Intel’s IXP2800–a state-of-the-art commercial Network Processor–in about 89 % of the deployments we consider. Further, in about 30 % of the deployments our platform improves the throughput by as much as 300%.

A Bloom filter is a simple randomized data structure that answers membership query with no false negative and a small false positive probability. It is an elegant data compression technique for membership information and has broad applications. In this paper, we generalize the traditional Bloom filter to Weighted Bloom Filter, which incorporates the information on the query frequencies and the membership likelihood of the elements into its optimal design. It has been widely observed that in many applications, some popular elements are queried much more often than the others. The traditional Bloom filter for data sets with irregular query patterns and non-uniform membership likelihood can be further optimized. We derive the optimal configuration of the Bloom filter with query-frequency and membershiplikelihood information, and show that the adapted Bloom filter always outperforms the traditional Bloom filter. Under reasonable frequency models such as the step distribution or the Zipf’s distribution, the improvement of the false positive probability of the weighted Bloom filter over that of the traditional Bloom filter has been evaluated by simulations.

The increasingly more stringent performance and power requirements of Internet routers call for scalable IP lookup strategies that go beyond the currently viable TCAM- and trie-based solutions. This paper describes a new hash-based IP lookup scheme that is both storage efficient and highperformance. In order to achieve high storage efficiency, we take a multi-hashing approach and employ an advanced hashing technique that effectively resolves hashing collisions by dynamically migrating IP prefixes that are already in the lookup table as new prefixes are inserted. To obtain high lookup throughput, the multiple hash tables are accessed in parallel (using different hash functions) or in a pipelined manner. We evaluate the proposed scheme using up-to-date core routing tables and discuss how its key design parameters can be determined. When compared with state-of-the-art TCAM designs, our scheme reduces area and power requirements by 60 % and 80 % respectively, while achieving competitive lookup rates. We expect that the proposed scheme scales well with the anticipated routing table sizes and technologies in the future. 1

As the Internet grows, both the number of rules in packet filtering databases and the number of prefixes in IP lookup tables inside the router are growing. The packet processing engine is a critical part of the Internet router as it is used to perform packet forwarding (PF) and packet classification (PC). In both applications, processing has to be at wire speed. It is common to use hash-based schemes in packet processing engines; however, the downside of classic hashing techniques such as overflow and worst case memory access time, has to be dealt with. Implementing hash tables using set associative memory has the property that each bucket of a hash table can be searched in one memory cycle outperforming the conventional Ternary CAMs in terms of power and scalability. In this paper we present “Progressive Hashing ” (PH), a general open addressing hash-based packet processing scheme for Internet routers using the set associative memory architecture. Our scheme is an extension of the multiple hashing scheme and is amendable to high-performance hardware implementation with low overflow and low memory access latency. We show by experimenting with real IP lookup tables and synthetic packet filtering databases that PH reduces the overflow over the multiple hashing. The proposed PH processing engine is estimated to achieve an average processing speed of 160 Gbps for the PC application and 320 Gbps for the PF application.

In enterprise and data center networks, the scalability of the data plane becomes increasingly challenging as forwarding tables and link speeds grow. Simply building switches with larger amounts of faster memory is not appealing, since high-speed memory is both expensive and power hungry. Implementing hash tables in SRAM is not appealing either because it requires significant overprovisioning to ensure that all forwarding table entries fit. Instead, we propose the BUFFALO architecture, which uses a small SRAM to store one Bloom filter of the addresses associated with each outgoing link. We provide a practical switch design leveraging flat addresses and shortest-path routing. BUF-FALO gracefully handles false positives without reducing the packet-forwarding rate, while guaranteeing that packets reach their destinations with bounded stretch with high probability. We tune the sizes of Bloom filters to minimize false positives for a given memory size. We also handle routing changes and dynamically adjust Bloom filter sizes using counting Bloom filters in slow memory. Our extensive analysis, simulation, and prototype implementation in kernel-level Click show that BUFFALO significantly reduces memory cost, increases the scalability of the data plane, and improves packet-forwarding performance.

While the problem of high performance packet classification has received a great deal of attention in recent years, the research community has yet to develop algorithmic methods that can overcome the drawbacks of TCAM-based solutions. This paper introduces a hybrid approach, which partitions the filter set into subsets that are easy to search efficiently. The partitioning strategy groups filters that are close to one another in tuple space [10], which makes it possible to use information from single field lookups to limit the number of subsets that must be searched. We can tradeoff running time against space consumption by adjusting the coarseness of the tuple space partition. We find that for two-dimensional filter sets, the method finds the bestmatching filter with just four hash probes while limiting the memory space expansion factor to about 2. We also introduce a novel method for Longest Prefix Matching (LPM), which we use as a component of the overall packet classification algorithm. Our LPM method uses a small amount of on-chip memory to speedup the search of an off-chip data structure, but uses significantly less on-chip memory than earlier methods based on Bloom filters.

ntop.org One problem encountered while monitoring gigabit networks, is the need to filter only those packets that are interesting for a given task while ignoring the others. Popular packet filtering technologies enable users to specify complex filters but do not usually allow multiple filters to be specified. This paper describes the design and implementation of a new dynamic packet filtering solution that allows users to specify several IP filters simultaneously with almost no packet loss even on high-loaded gigabit links. The advantage is that modern traffic monitoring applications such as P2P, IPTV, VoIP monitoring, and lawful interception can dynamically set packet filters to efficiently discard packets into the operating system kernel according to traffic, calls and users being monitored.

Network management applications require accurate estimates of a wide range of flow-level traffic metrics. Given the inadequacy of current packet-sampling-based solutions, several application-specific monitoring algorithms have emerged. While these provide better accuracy for the specific applications they target, they increase router complexity and require vendors to commit to hardware primitives without knowing how useful they will be to meet the needs of future applications. In this paper, we show using trace-driven evaluations that such complexity and early commitment may not be necessary. We revisit the case for a “minimalist ” approach in which a small number of simple yet generic router primitives collect flow-level data from which different traffic metrics can be estimated. We demonstrate the feasibility and promise of such a minimalist approach using flow sampling and sample-and-hold as sampling primitives and configuring these in a network-wide coordinated fashion using cSamp. We show that this proposal yields better accuracy across a collection of application-level metrics than dividing the same memory resources across metric-specific algorithms. Moreover, because a minimalist approach enables late binding to what applicationlevel metrics are important, it better insulates router implementations and deployments from changing monitoring needs.