Results 1 
8 of
8
Toward a unified theory of sparse dimensionality reduction in Euclidean space
"... Let Φ ∈ Rm×n be a sparse JohnsonLindenstrauss transform [KN14] with s nonzeroes per column. For a subset T of the unit sphere, ε ∈ (0, 1/2) given, we study settings for m, s required to ensure E Φ sup x∈T ∣∣‖Φx‖22 − 1∣ ∣ < ε, i.e. so that Φ preserves the norm of every x ∈ T simultaneously and ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
Let Φ ∈ Rm×n be a sparse JohnsonLindenstrauss transform [KN14] with s nonzeroes per column. For a subset T of the unit sphere, ε ∈ (0, 1/2) given, we study settings for m, s required to ensure E Φ sup x∈T ∣∣‖Φx‖22 − 1∣ ∣ < ε, i.e. so that Φ preserves the norm of every x ∈ T simultaneously and multiplicatively up to 1 + ε. We introduce a new complexity parameter, which depends on the geometry of T, and show that it suffices to choose s and m such that this parameter is small. Our result is a sparse analog of Gordon’s theorem, which was concerned with a dense Φ having i.i.d. Gaussian entries. We qualitatively unify several results related to the JohnsonLindenstrauss lemma, subspace embeddings, and Fourierbased restricted isometries. Our work also
Sparsity lower bounds for dimensionality reducing maps
 In arXiv:1211.0995v1
, 2012
"... We give neartight lower bounds for the sparsity required in several dimensionality reducing linear maps. First, consider the JohnsonLindenstrauss (JL) lemma which states that for any set of n vectors in R d there is a matrix A ∈ R m×d with m = O(ε −2 log n) such that mapping by A preserves pairwis ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
We give neartight lower bounds for the sparsity required in several dimensionality reducing linear maps. First, consider the JohnsonLindenstrauss (JL) lemma which states that for any set of n vectors in R d there is a matrix A ∈ R m×d with m = O(ε −2 log n) such that mapping by A preserves pairwise Euclidean distances of these n vectors up to a 1±ε factor. We show that there exists a set of n vectors such that any such matrix A with at most s nonzero entries per column must have s = Ω(ε −1 log n / log(1/ε)) as long as m < O(n / log(1/ε)). This bound improves the lower bound of Ω(min{ε −2, ε −1 √ log m d}) by [DasguptaKumarSarlós, STOC 2010], which only held against the stronger property of distributional JL, and only against a certain restricted class of distributions. Meanwhile our lower bound is against the JL lemma itself, with no restrictions. Our lower bound matches the sparse JohnsonLindenstrauss upper bound of [KaneNelson, SODA 2012] up to an O(log(1/ε)) factor. Next, we show that any m×n matrix with the krestricted isometry property (RIP) with constant distortion must have at least Ω(k log(n/k)) nonzeroes per column if m = O(k log(n/k)), the optimal number of rows of RIP matrices, and k < n / polylog n. This improves the previous lower bound of Ω(min{k, n/m}) by [Chandar, 2010] and shows that for virtually all k it is impossible to have a sparse RIP matrix with an optimal number of rows. Both lower bounds above also offer a tradeoff between sparsity and the number of rows. Lastly, we show that any oblivious distribution over subspace embedding matrices with 1 nonzero per column and preserving distances in a d dimensionalsubspace up to a constant factor must have at least Ω(d2) rows. This matches one of the upper bounds in [NelsonNguy˜ên, 2012] and shows the impossibility of obtaining the best of both of constructions in that work, namely 1 nonzero per column and Õ(d) rows. 1
Local recovery properties of capacity achieving codes
 In Information Theory and Applications Workshop (ITA), 2013
, 2013
"... AbstractA code is called locally recoverable or repairable if any symbol of a codeword can be recovered by reading only a small (constant) number of other symbols. The notion of local recoverability is important in the area of distributed storage where a most frequent errorevent is a single stora ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
AbstractA code is called locally recoverable or repairable if any symbol of a codeword can be recovered by reading only a small (constant) number of other symbols. The notion of local recoverability is important in the area of distributed storage where a most frequent errorevent is a single storage node failure. A common objective is to repair the node by downloading data from as few other storage node as possible. In this paper we study the basic errorcorrecting properties of a locally recoverable code. We provide tight upper and lower bound on the localrecoverability of a code that achieves capacity of a symmetric channel. In particular it is shown that, if the coderate is less than the capacity then for the optimal codes, the maximum number of codeword symbols required to recover one lost symbol must scale as log 1 .
Practical Compression with ModelCode Separation
"... Abstract Two aspects of compression data modeling and coding are not always conceived as distinct, nor implemented as such in current compression systems, leading to difficulties of an architectural nature. This work contributes an alternative "modelcode separation" architecture for co ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract Two aspects of compression data modeling and coding are not always conceived as distinct, nor implemented as such in current compression systems, leading to difficulties of an architectural nature. This work contributes an alternative "modelcode separation" architecture for compression, based on iterative messagepassing algorithms over graphical models representing the modeling and coding aspects of compression. Systems following this architecture resolve important challenges posed by current systems, and stand to benefit further from advances in the understanding of data and the algorithms that process them.
Local Recovery in Data Compression for General Sources
"... Abstract—Source coding is concerned with optimally compressing data, so that it can be reconstructed up to a specified distortion from its compressed representation. Usually, in fixedlength compression, a sequence of n symbols (from some alphabet) is encoded to a sequence of k symbols (bits). The ..."
Abstract
 Add to MetaCart
Abstract—Source coding is concerned with optimally compressing data, so that it can be reconstructed up to a specified distortion from its compressed representation. Usually, in fixedlength compression, a sequence of n symbols (from some alphabet) is encoded to a sequence of k symbols (bits). The decoder produces an estimate of the original sequence of n symbols from the encoded bits. The ratedistortion function characterizes the optimal possible rate of compression allowing a given distortion in reconstruction as n grows. This function depends on the source probability distribution. In a locally recoverable decoding, to reconstruct a single symbol, only a few compressed bits are accessed. In this paper we find the limits of local recovery for rates near the ratedistortion function. For a wide set of source distributions, we show that, it is possible to compress within of the ratedistortion function such the local recoverability grows as Ω(log ( 1)); that is, in order to recover one source symbol, at least Ω(log ( 1 bits of the compressed symbols are queried. We also show order optimal impossibility results. Similar results are provided for lossless source coding as well. I.
MIT CSAIL
, 2014
"... The Restricted Isometry Property (RIP) is a fundamental property of a matrix which enables sparse recovery. Informally, an m × n matrix satisfies RIP of order k for the üp norm, if ‖Ax‖p ≈ ‖x‖p for every x with at most k nonzero coordinates. For every 1 ≤ p < ∞ we obtain almost tight bounds on t ..."
Abstract
 Add to MetaCart
(Show Context)
The Restricted Isometry Property (RIP) is a fundamental property of a matrix which enables sparse recovery. Informally, an m × n matrix satisfies RIP of order k for the üp norm, if ‖Ax‖p ≈ ‖x‖p for every x with at most k nonzero coordinates. For every 1 ≤ p < ∞ we obtain almost tight bounds on the minimum number of rows m necessary for the RIP property to hold. Before, only the cases p ∈ {1, 1 + 1log k, 2} were studied. Interestingly, our results show that the case p = 2 is a ‘singularity ’ in terms of the optimum value of m. We also obtain almost tight bounds for the column sparsity of RIP matrices and discuss implications of our results for the Stable Sparse Recovery problem. ar X iv
UpdateEfficiency and Local Repairability Limits for Capacity Approaching Codes
"... Abstract—Motivated by distributed storage applications, we investigate the degree to which capacity achieving codes can be efficiently updated when a single information symbol changes, and the degree to which such codes can be efficiently repaired when a single encoded symbol is lost. Specifically, ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—Motivated by distributed storage applications, we investigate the degree to which capacity achieving codes can be efficiently updated when a single information symbol changes, and the degree to which such codes can be efficiently repaired when a single encoded symbol is lost. Specifically, we first develop conditions under which optimum errorcorrection and updateefficiency are possible. We establish that the number of encoded bits that should change in response to a change in a single information bit must scale logarithmically in the blocklength of the code, if we are to achieve any nontrivial rate with vanishing probability of error over the binary erasure or binary symmetric channels. Moreover, we show that there exist capacityachieving codes with this scaling. With respect to local repairability, we develop tight upper and lower bounds on the number of remaining encoded bits that are needed to recover a single lost encoded bit. In particular, we show that when the rate of an optimal code is ɛ below capacity, the maximum number of codeword symbols required to recover one lost symbol must scale as log 1/ɛ. Several variations on—and extensions of—these results are also developed, including to the problem of ratedistortion coding. Index Terms—errorcorrecting codes, linear codes, updateefficiency, locally repairable codes, lowdensity generator matrix codes, lowdensity parity check codes, channel capacity I.