#### DMCA

## Locality and availability in distributed storage (2014)

Venue: | IN PROCEEDINGS OF THE IEEE INTERNATIONAL SYMPOSIUM ON INFORMATION THEORY (ISIT |

Citations: | 3 - 1 self |

### Citations

333 | Network Coding for Distributed Storage Systems
- Dimakis, Godfrey, et al.
- 2007
(Show Context)
Citation Context ...ock replication to traditional and modern storage codes. It is now well understood that classical codes (such as Reed-Solomon) are highly suboptimal for distributed settings due to the repair problem =-=[1]-=-. Several storage codes have been recently The authors are with the Dept. of ECE, The University of Texas at Austin, Austin, TX 78751 USA. E-mail: {ankitsr, dimitris}@utexas.edu, {dimakis, sriram}@aus... |

161 |
Theory of codes with maximum rank distance
- Gabidulin
- 1985
(Show Context)
Citation Context ...ther in exactly λ = 1 block. The blocks from each of the 7 parallel classes E1, . . . , E7 partition the set X . B. Gabidulin codes Gabidulin codes are an example of maximum rank distance (MRD) codes =-=[32]-=-. Gabidulin codes are MDS. Encoding m = (m1,m2, . . . ,mK) ∈ FKqM to a codeword c of an [N ,K,D] Gabidulin code over FqM is performed by evaluating a data polynomial f (y) = ∑K i=1miy qi−1 at N -linea... |

105 | Optimal exact-regenerating codes for distributed storage at the MSR and MBR points via a productmatrix construction - Rashmi, Shah, et al. - 2011 |

81 | Pyramid Codes: Flexible Schemes to Trade Space for Access Efficiency in Reliable Data Storage Systems
- Huang, Chen, et al.
- 2007
(Show Context)
Citation Context ...nt explicit constructions for codes with (r, t)-availability, and analyze their minimum distance. In particular, in Sec. IV-C we design (n, k, r, t)-LRCs by modifying the Pyramid code construction of =-=[9]-=-. Then, in Sec. IV-D we use Gabidulin codes in order to obtain (n, k, r, t)-LRCs with all-symbol locality. Finally, Sec. IV-E describes the role of resolvable designs in our proposed constructions. No... |

64 | Locally Repairable Codes
- Papailiopoulos, Dimakis
- 2012
(Show Context)
Citation Context ...er than that of (10). Therefore, we can combine the two cases to obtain the bound in (6). Remark 1. Note that for t = 1, the bound in (6) reduces to the distance bound for codes with r-locality [10], =-=[12]-=-, i.e., dmin(C) ≤ n− k − ⌈ k r ⌉ + 2. Next, we present a more general bound on the minimum distance of an (n, k, r, t)-LRC. This bound does not assume linearity of a code or that the repair groups ass... |

59 | XORing Elephants: Novel Erasure Codes for Big Data
- Sathiamoorthy, Asteris, et al.
- 2013
(Show Context)
Citation Context .... Some of these results have found their way into practice: codes with small locality were recently deployed in Azure production clusters [20], while others have been tested in Facebook clusters [6], =-=[21]-=-. Code designs with small repair bandwidth and locality are attractive for archival and cold data. This is information that is rarely accessed or modified, usually involving back-end systems that stor... |

54 | Erasure coding in windows azure storage.
- Huang, Simitci, et al.
- 2012
(Show Context)
Citation Context ... process, a quantity called locality, were studied in [8]–[19]. Some of these results have found their way into practice: codes with small locality were recently deployed in Azure production clusters =-=[20]-=-, while others have been tested in Facebook clusters [6], [21]. Code designs with small repair bandwidth and locality are attractive for archival and cold data. This is information that is rarely acce... |

49 | On the locality of codeword symbols,”
- Gopalan, Huang, et al.
- 2012
(Show Context)
Citation Context ... greater than that of (10). Therefore, we can combine the two cases to obtain the bound in (6). Remark 1. Note that for t = 1, the bound in (6) reduces to the distance bound for codes with r-locality =-=[10]-=-, [12], i.e., dmin(C) ≤ n− k − ⌈ k r ⌉ + 2. Next, we present a more general bound on the minimum distance of an (n, k, r, t)-LRC. This bound does not assume linearity of a code or that the repair grou... |

41 | Repair optimal erasure codes through Hadamard designs - Papailiopoulos, Dimakis, et al. - 2011 |

38 | Simple Regenerating Codes: Network Coding for Cloud Storage - Papailiopoulos, Luo, et al. - 2012 |

31 | Optimal locally repairable and secure codes for distributed storage systems,”
- Rawat, Koyluoglu, et al.
- 2014
(Show Context)
Citation Context ...codes [32] to construct (n, k, r, t)- LRCs with all-symbol locality when r|k, r|N , and Assumption 1 holds. Gabidulin codes have been previously used to obtain codes with all-symbol locality in [14], =-=[15]-=-. 1) Construction II: Let GGab be a generator matrix of an [N = N + t− 1,K = k] Gabidulin code. We transform GGab into ḠGab = ( G1Gab )−1 GGab = [I | ( G1Gab )−1 G2Gab], which is a generator matrix o... |

29 | Optimal Linear Codes with a Local-Error-Correction Property - Prakash, Kamath, et al. - 2012 |

27 | Zigzag codes: MDS array codes with optimal rebuilding
- Tamo, Wang, et al.
- 2011
(Show Context)
Citation Context ... optimize the number of bits communicated during repairs (a quantity called repair bandwidth) were developed, for example, in [1]–[6] and references therein. Codes with small disk-I/O were studied in =-=[3]-=-, [7]. Finally, codes that minimize the number of nodes that participate in the repair process, a quantity called locality, were studied in [8]–[19]. Some of these results have found their way into pr... |

24 | Batch codes and their applications.
- Ishai, Kushilevitz, et al.
- 2004
(Show Context)
Citation Context ... bound in the asymptotically zero rate regime. The tightness of this bound is open in the general case. We briefly discuss their bound in Sec. III, where we extend it to non-linear codes. Batch codes =-=[29]-=- also enable parallel reads, in an even stronger sense of allowing multiple reads of different blocks simultaneously. However, current work on Batch codes does not consider fault tolerance. It is inte... |

21 | In search of I/O-optimal recovery from disk failures.
- Khan, Burns, et al.
- 2011
(Show Context)
Citation Context ...mize the number of bits communicated during repairs (a quantity called repair bandwidth) were developed, for example, in [1]–[6] and references therein. Codes with small disk-I/O were studied in [3], =-=[7]-=-. Finally, codes that minimize the number of nodes that participate in the repair process, a quantity called locality, were studied in [8]–[19]. Some of these results have found their way into practic... |

20 | A solution to the network challenges of data recovery in erasure-coded distributed storage systems: A study on the facebook warehouse cluster.
- Rashmi, Shah, et al.
- 2013
(Show Context)
Citation Context ...veloped, each optimized for a different repair cost metric. Codes that optimize the number of bits communicated during repairs (a quantity called repair bandwidth) were developed, for example, in [1]–=-=[6]-=- and references therein. Codes with small disk-I/O were studied in [3], [7]. Finally, codes that minimize the number of nodes that participate in the repair process, a quantity called locality, were s... |

18 | Optimal locally repairable codes and connections to matroid theory - Tamo, Papailiopoulos, et al. |

17 | Codes with local regeneration - Kamath, Prakash, et al. - 2012 |

16 | Permutation code: optimal exact-repair of a single failed node in MDS code based distributed storage systems - Cadambe, Huang, et al. - 2011 |

12 | Reliable Memories with Subline Accesses
- Han, Lastras-Montano
- 2007
(Show Context)
Citation Context ...ences therein. Codes with small disk-I/O were studied in [3], [7]. Finally, codes that minimize the number of nodes that participate in the repair process, a quantity called locality, were studied in =-=[8]-=-–[19]. Some of these results have found their way into practice: codes with small locality were recently deployed in Azure production clusters [20], while others have been tested in Facebook clusters ... |

12 | On the delay-storage trade-off in content download from coded distributed storage systems,” arXiv preprint arXiv:1305.3945, - Joshi, Liu, et al. - 2013 |

9 | Error resilience in distributed storage via rankmetric codes
- Silberstein, Rawat, et al.
- 2012
(Show Context)
Citation Context ...dulin codes [32] to construct (n, k, r, t)- LRCs with all-symbol locality when r|k, r|N , and Assumption 1 holds. Gabidulin codes have been previously used to obtain codes with all-symbol locality in =-=[14]-=-, [15]. 1) Construction II: Let GGab be a generator matrix of an [N = N + t− 1,K = k] Gabidulin code. We transform GGab into ḠGab = ( G1Gab )−1 GGab = [I | ( G1Gab )−1 G2Gab], which is a generator ma... |

9 | A Mazumdar. An upper bound on the size of locally recoverable codes
- Cadambe
- 2013
(Show Context)
Citation Context ...s therein. Codes with small disk-I/O were studied in [3], [7]. Finally, codes that minimize the number of nodes that participate in the repair process, a quantity called locality, were studied in [8]–=-=[19]-=-. Some of these results have found their way into practice: codes with small locality were recently deployed in Azure production clusters [20], while others have been tested in Facebook clusters [6], ... |

9 | When do redundant requests reduce latency?” arXiv preprint arXiv:1311.2851,
- Shah, Lee, et al.
- 2013
(Show Context)
Citation Context ...es concurrently by multiple system jobs. To the best of our knowledge, there has been little work on the potential benefits of coding for hot data. A notable exception is the recent line of work from =-=[22]-=-–[24] and references therein that combines queuing theory with coding theory for distributed storage systems. In this paper we explore the orthogonal direction of providing multiple ways to reconstruc... |

6 | On the locality of codeword symbols in non-linear codes,” Arxiv 1303.3921, - Forbes, Yekhanin - 2013 |

6 | Locally repairable codes with multiple repair alternatives
- Pamies-Juarez, Hollmann, et al.
- 2013
(Show Context)
Citation Context ...d high availability, but are arbitrarily high-rate, and have distance asymptotically equal to that of an (n, k) maximum-distance separable (MDS) code. Prior work on codes with (r, t)-availability: In =-=[25]-=-, PamiesJuarez et al. use projective geometries to construct good codes that enable multiple disjoint repair groups. Asteris et al. study the availability for repairable fountain codes in [26]. Tamo e... |

6 | A family of optimal locally recoverable codes
- Tamo, Barg
- 2014
(Show Context)
Citation Context ...ble multiple disjoint repair groups. Asteris et al. study the availability for repairable fountain codes in [26]. Tamo et al. recently constructed codes with good locality and availability properties =-=[27]-=-. In a parallel work, Wang et al. present a more general upper bound on minimum distance of linear codes with (r, t)-availability [28]. They further show the existence of codes that achieve the bound ... |

5 |
Repairable fountain codes,”
- Asteris, Dimakis
- 2014
(Show Context)
Citation Context ...ity: In [25], PamiesJuarez et al. use projective geometries to construct good codes that enable multiple disjoint repair groups. Asteris et al. study the availability for repairable fountain codes in =-=[26]-=-. Tamo et al. recently constructed codes with good locality and availability properties [27]. In a parallel work, Wang et al. present a more general upper bound on minimum distance of linear codes wit... |

4 | Repair locality with multiple erasure tolerance
- Wang, Zhang
(Show Context)
Citation Context ... constructed codes with good locality and availability properties [27]. In a parallel work, Wang et al. present a more general upper bound on minimum distance of linear codes with (r, t)-availability =-=[28]-=-. They further show the existence of codes that achieve the bound in the asymptotically zero rate regime. The tightness of this bound is open in the general case. We briefly discuss their bound in Sec... |

2 | Multipermutation codes in the Ulam metric for nonvolatile memories
- Farnoud, Milenkovic
(Show Context)
Citation Context ...y. Finally, Sec. IV-E describes the role of resolvable designs in our proposed constructions. Note that the resolvable designs have been previously used to construct codes in other settings, e.g. see =-=[30]-=-. Before describing the code constructions, we present in Sec. IV-A and Sec. IV-B a brief introduction to resolvable designs and Gabidulin codes. A. Background on resolvable designs Here, we briefly i... |

1 | Tofec: Achieving optimal throughput-delay trade-off of cloud storage using erasure codes
- Liang, Kozat
(Show Context)
Citation Context ...ncurrently by multiple system jobs. To the best of our knowledge, there has been little work on the potential benefits of coding for hot data. A notable exception is the recent line of work from [22]–=-=[24]-=- and references therein that combines queuing theory with coding theory for distributed storage systems. In this paper we explore the orthogonal direction of providing multiple ways to reconstruct a s... |