Results 1 - 10
of
14
Programming Your Network at Run-time for Big Data Applications
"... Recent advances of software defined networking and optical switching technology make it possible to program the network stack all the way from physical topology to flow level traffic control. In this paper, we leverage the combination of SDN controller with optical switching to explore the tight int ..."
Abstract
-
Cited by 20 (3 self)
- Add to MetaCart
(Show Context)
Recent advances of software defined networking and optical switching technology make it possible to program the network stack all the way from physical topology to flow level traffic control. In this paper, we leverage the combination of SDN controller with optical switching to explore the tight integration of application and network control. We particularly study the run-time network configuration for big data applications to jointly optimize application performance and network utilization. We use Hadoop as an example to discuss the integrated network control architecture, job scheduling, topology and routing configuration mechanisms for Hadoop jobs. Our analysis suggests that such an integrated control has great potential to improve application performance with relatively small configuration overhead. We believe our study shows early promise of achieving the long-term goal of tight network and application integration using SDN.
Proteus: A Topology Malleable Data Center Network
- in Proc. ACM HotNets
, 2010
"... Full-bandwidth connectivity between all servers of a data center may be necessary for all-to-all traffic patterns, but such interconnects suffer from high cost, complexity, and energy consumption. Recent work has argued that if allto-all traffic is uncommon, oversubscribed network architectures that ..."
Abstract
-
Cited by 16 (7 self)
- Add to MetaCart
(Show Context)
Full-bandwidth connectivity between all servers of a data center may be necessary for all-to-all traffic patterns, but such interconnects suffer from high cost, complexity, and energy consumption. Recent work has argued that if allto-all traffic is uncommon, oversubscribed network architectures that can adapt the topology to meet traffic demands, are sufficient. In line with this work, we propose Proteus1, an all-optical architecture targeting unprecedented topologyflexibility, lower complexity and higher energy efficiency.
A Taxonomy and Survey on Green Data Center Networks,” Future Generation Computer Systems
"... ..."
(Show Context)
ECHO: Recreating Network Traffic Maps for Datacenters with Tens of Thousands of Servers
"... Abstract—Large-scale datacenters now host a large part of the world’s data and computation, which makes their design a crucial architectural challenge. Datacenter (DC) applications, unlike traditional workloads, are dominated by user patterns that only emerge in the large-scale. This creates the nee ..."
Abstract
-
Cited by 5 (3 self)
- Add to MetaCart
(Show Context)
Abstract—Large-scale datacenters now host a large part of the world’s data and computation, which makes their design a crucial architectural challenge. Datacenter (DC) applications, unlike traditional workloads, are dominated by user patterns that only emerge in the large-scale. This creates the need for concise, accurate and scalable analytical models that capture both their temporal and spatial features and can be used to create representative activity patterns. Unfortunately, previous work lacks the ability to track the complex patterns that are present in these applications, or scales poorly with the size of the system. In this work, we focus on the network aspect of datacenter workloads. We present ECHO, a scalable and accurate modeling scheme that uses hierarchical Markov Chains to capture the network activity of large-scale applications in time and space. ECHO can also use these models to re-create representative network traffic patterns. We validate the model against real DC-scale applications, such as Websearch and show marginal deviations between original and generated workloads. We verify that ECHO captures all the critical features of DC workloads, such as the locality of communication and burstiness and evaluate the granularity necessary for this. Finally we perform a detailed characterization of the network traffic for workloads in DCs of tens of thousands of servers over significant time frames. I.
VirtualKnotter: Online Virtual Machine Shuffling for Congestion Resolving in Virtualized Datacenter
"... Abstract—Our measurements on production datacenter traffic together with recently-reported results [1] suggest that datacenter networks suffer from long-lived congestion caused by core network oversubscription and unbalanced workload placement. In contrast to traditional traffic engineering approach ..."
Abstract
-
Cited by 5 (1 self)
- Add to MetaCart
(Show Context)
Abstract—Our measurements on production datacenter traffic together with recently-reported results [1] suggest that datacenter networks suffer from long-lived congestion caused by core network oversubscription and unbalanced workload placement. In contrast to traditional traffic engineering approaches that optimize flow routing, in this paper, we explore the opportunity to address the continuous congestion via optimizing VM placement in virtualized datacenters. To this end, we present VirtualKnotter, an efficient online VM placement algorithm to reduce congestion with controllable VM migration traffic as well as low time complexity. Our evaluation with both real and synthetic traffic patterns shows that VirtualKnotter performs close to the baseline algorithm in terms of link unitization, with only 5%-10 % migration traffic of the baseline algorithm. Furthermore, VirtualKnotter decreases link congestion time by 53 % for the production datacenter traffic. I.
How to Split a Flow?
"... Many practically deployed flow algorithms produce the output as a set of values associated with the network links. However, to actually deploy a flow in a network we often need to represent it as a set of paths between the source and destination nodes. In this paper we consider the problem of decom ..."
Abstract
-
Cited by 5 (0 self)
- Add to MetaCart
(Show Context)
Many practically deployed flow algorithms produce the output as a set of values associated with the network links. However, to actually deploy a flow in a network we often need to represent it as a set of paths between the source and destination nodes. In this paper we consider the problem of decomposing a flow into a small number of paths. We show that there is some fixed constant β> 1 such that it is NP-hard to find a decomposition in which the number of paths is larger than the optimal by a factor of at most β. Furthermore, this holds even if arcs are associated only with three different flow values. We also show that straightforward greedy algorithms for the problem can produce much larger decompositions than the optimal one, on certain well tailored inputs. On the positive side we present a new approximation algorithm that decomposes all but an ɛ-fraction of the flow into at most O(1/ɛ 2) times the smallest possible number of paths. We compare the decompositions produced by these algorithms on real production networks and on synthetically generated data. Our results indicate that the dependency of the decomposition size on the fraction of flow covered is exponential. Hence, covering the last few percent of the flow may be costly, so if the application allows, it may be a good idea to decompose most but not all the flow. The experiments also reveal the fact that while for realistic data the greedy approach works very well, our novel algorithm which has a provable worst case guarantee, typically produces only slightly larger decompositions.
Depth-first worst-fit search based multipath routing for data center networks
- IEEE Global Communications Conference (GLOBECOM
, 2012
"... Abstract-Modern data center networks (DCNs) often use multi-rooted topologies, which offer multipath capability, for increased bandwidth and fault tolerance. However, traditional routing algorithms for the Internet have no or limited support for multipath routing, and cannot fully utilize available ..."
Abstract
-
Cited by 2 (1 self)
- Add to MetaCart
(Show Context)
Abstract-Modern data center networks (DCNs) often use multi-rooted topologies, which offer multipath capability, for increased bandwidth and fault tolerance. However, traditional routing algorithms for the Internet have no or limited support for multipath routing, and cannot fully utilize available bandwidth in such DCNs. In this paper, we study the multipath routing problem for DCNs. We first formulate the problem as an integer linear program, but it is not suitable for fast on-the-fly route calculation. For a practical solution, we propose the Depth-First Worst-Fit Search based multipath routing algorithm. The main idea is to use depth-first search to find a sequence of worst-fit links to connect the source and destination of a flow. Since DCN topologies are usually hierarchical, our algorithm uses depthfirst search to quickly traverse between hierarchical layers to find a path. When there are multiple links to a neighboring layer, the worst-fit link selection criterion enables the algorithm to make the selection decision with constant time complexity by leveraging the max-heap data structure, and use a small number of selections to find all the links of a path. Further, worst-fit also achieves load balancing, and thus generates low queueing delay, which is a major component of the end-to-end delay. We have evaluated the proposed algorithm by extensive simulations, and compared its average number of link selections and average end-to-end delay with competing solutions. The simulation results fully demonstrate the superiority of our algorithm and validate the effectiveness of our designs.
Revisiting Flow-Based Load Balancing: Stateless Path Selection in Data Center Networks
"... Hash-based load-balancing techniques are widely used to distribute the load over multiple forwarding paths and preserve the packet sequence of transport-level flows. Forcing a long-lived, i.e., elephant, flow to follow a specific path in the network is a desired mechanism in data center networks to ..."
Abstract
- Add to MetaCart
(Show Context)
Hash-based load-balancing techniques are widely used to distribute the load over multiple forwarding paths and preserve the packet sequence of transport-level flows. Forcing a long-lived, i.e., elephant, flow to follow a specific path in the network is a desired mechanism in data center networks to avoid crossing hot spots. This limits the formation of bottlenecks and so improves the network use. Unfortunately, current per-flow load-balancing methods do not allow sources to deterministically force a specific path for a flow. In this paper, we propose a deterministic approach enabling end hosts to steer their flows over any desired load-balanced path without relying on any packet header extension. By using an invertible mechanism instead of solely relying on a hash function in routers, our method allows to easily select the packet’s header field values in order to force the selection of a given load-balanced path without storing any state in routers. We perform various simulations and experiments to evaluate the performance and prove the feasibility of our method using a Linux kernel implementation. Furthermore, we demonstrate with simulations and lab experiments how MultiPath TCP can benefit from the combination of our solution with a flow scheduling system that efficiently distributes elephant flows in large data center networks. Keywords: Load Balancing; Data Center Networks; Multipath TCP
This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination. IEEE/ACM TRANSACTIONS ON NETWORKING 1 OSA: An Optical Switching Architecture for Data Center Networks With Unp
"... Abstract—A detailed examination of evolving traffic characteristics, operator requirements, and network technology trends suggests a move away from nonblocking interconnects in data center networks (DCNs). As a result, recent efforts have advocated oversubscribed networks with the capability to adap ..."
Abstract
- Add to MetaCart
(Show Context)
Abstract—A detailed examination of evolving traffic characteristics, operator requirements, and network technology trends suggests a move away from nonblocking interconnects in data center networks (DCNs). As a result, recent efforts have advocated oversubscribed networks with the capability to adapt to trafficrequirements on-demand. In this paper, we present the design, implementation, and evaluation of OSA, a novel Optical Switching Architecture for DCNs. Leveraging runtime reconfigurable optical devices, OSA dynamically changes its topology and link capacities, thereby achieving unprecedented flexibility to adapt to dynamic traffic patterns. Extensive analytical simulations using both real and synthetic traffic patterns demonstrate that OSA can deliver high bisection bandwidth (60%–100 % of the nonblocking architecture). Implementation and evaluation of a small-scale functional prototype further demonstrate the feasibility of OSA. Index Terms—Data center networks (DCNs), optical networking technology, switching architecture. I.
Luis Chiang
"... Networks today rely on middleboxes to provide critical performance, security, and policy compliance capabilities. Achieving these benefits and ensuring that the traffic is directed through the desired sequence of middleboxes requires significant manual effort and operator expertise. In this respect, ..."
Abstract
- Add to MetaCart
(Show Context)
Networks today rely on middleboxes to provide critical performance, security, and policy compliance capabilities. Achieving these benefits and ensuring that the traffic is directed through the desired sequence of middleboxes requires significant manual effort and operator expertise. In this respect, Software-Defined Networking (SDN) offers a promising alternative. Middleboxes, however, introduce new aspects (e.g., policy composition, resource management, packet modifications) that fall outside the purvey of traditional L2/L3 functions that SDN supports (e.g., access control or routing). This paper presents SIMPLE, a SDN-based policy enforcement layer for efficient middlebox-specific “traffic steering”. In designing SIMPLE, we take an explicit stance to work within the constraints of legacy middleboxes and existing SDN interfaces. To this end, we address algorithmic and system design challenges to demonstrate the feasibility of using SDN to simplify middlebox traffic steering. In doing so, we also take a significant step toward addressing industry concerns surrounding the ability of SDN to integrate with existing infrastructure and support L4–L7 capabilities.
