Results 1  10
of
84
Onebit compressed sensing by linear programming
, 2011
"... We give the first computationally tractable and almost optimal solution to the problem of onebit compressed sensing, showing how to accurately recover an ssparse vector x ∈ R n from the signs of O(s log² (n/s)) random linear measurements of x. The recovery is achieved by a simple linear program. ..."
Abstract

Cited by 57 (5 self)
 Add to MetaCart
We give the first computationally tractable and almost optimal solution to the problem of onebit compressed sensing, showing how to accurately recover an ssparse vector x ∈ R n from the signs of O(s log² (n/s)) random linear measurements of x. The recovery is achieved by a simple linear program. This result extends to approximately sparse vectors x. Our result is universal in the sense that with high probability, one measurement scheme will successfully recover all sparse vectors simultaneously. The argument is based on solving an equivalent geometric problem on random hyperplane tessellations.
Robust 1bit compressed sensing and sparse logistic regression: A convex programming approach. Preprint. Available at http://arxiv.org/abs/1202.1212
"... Abstract. This paper develops theoretical results regarding noisy 1bit compressed sensing and sparse binomial regression. Wedemonstrate thatasingle convexprogram gives anaccurate estimate of the signal, or coefficient vector, for both of these models. We show that an ssparse signal in R n can be a ..."
Abstract

Cited by 44 (4 self)
 Add to MetaCart
(Show Context)
Abstract. This paper develops theoretical results regarding noisy 1bit compressed sensing and sparse binomial regression. Wedemonstrate thatasingle convexprogram gives anaccurate estimate of the signal, or coefficient vector, for both of these models. We show that an ssparse signal in R n can be accurately estimated from m = O(slog(n/s)) singlebit measurements using a simple convex program. This remains true even if each measurement bit is flipped with probability nearly 1/2. Worstcase (adversarial) noise can also be accounted for, and uniform results that hold for all sparse inputs are derived as well. In the terminology of sparse logistic regression, we show that O(slog(2n/s)) Bernoulli trials are sufficient to estimate a coefficient vector in R n which is approximately ssparse. Moreover, the same convex program works for virtually all generalized linear models, in which the link function may be unknown. To our knowledge, these are the first results that tie together the theory of sparse logistic regression to 1bit compressed sensing. Our results apply to general signal structures aside from sparsity; one only needs to know the size of the set K where signals reside. The size is given by the mean width of K, a computable quantity whose square serves as a robust extension of the dimension. 1.
Regime Change: BitDepth versus MeasurementRate in Compressive Sensing
, 2011
"... The recently introduced compressive sensing (CS) framework enables digital signal acquisition systems to take advantage of signal structures beyond bandlimitedness. Indeed, the number of CS measurements required for stable reconstruction is closer to the order of the signal complexity than the Nyqui ..."
Abstract

Cited by 32 (1 self)
 Add to MetaCart
(Show Context)
The recently introduced compressive sensing (CS) framework enables digital signal acquisition systems to take advantage of signal structures beyond bandlimitedness. Indeed, the number of CS measurements required for stable reconstruction is closer to the order of the signal complexity than the Nyquist rate. To date, the CS theory has focused on realvalued measurements, but in practice, measurements are mapped to bits from a finite alphabet. Moreover, in many potential applications the total number of measurement bits is constrained, which suggests a tradeoff between the number of measurements and the number of bits per measurement. We study this situation in this paper and show that there exist two distinct regimes of operation that correspond to high/low signaltonoise ratio (SNR). In the measurement compression (MC) regime, a high SNR favors acquiring fewer measurements with more bits per measurement; in the quantization compression (QC) regime, a low SNR favors acquiring more measurements with fewer bits per measurement. A surprise from our analysis and experiments is that in many practical applications it is better to operate in the QC regime, even acquiring as few as 1 bit per measurement.
Universal RateEfficient Scalar Quantization
 IEEE TRANSACTIONS ON INFORMATION THEORY, TO APPEAR
, 2011
"... Scalar quantization is the most practical and straightforward approach to signal quantization. However, it has been shown that scalar quantization of oversampled or compressively sensed signals can be inefficient in terms of the ratedistortion tradeoff, especially as the oversampling rate or the s ..."
Abstract

Cited by 31 (9 self)
 Add to MetaCart
Scalar quantization is the most practical and straightforward approach to signal quantization. However, it has been shown that scalar quantization of oversampled or compressively sensed signals can be inefficient in terms of the ratedistortion tradeoff, especially as the oversampling rate or the sparsity of the signal increases. In this paper, we modify the scalar quantizer to have discontinuous quantization regions. We demonstrate that with this modification it is possible to achieve exponential decay of the quantization error as a function of the oversampling rate instead of the quadratic decay exhibited by current approaches. Our approach is universal in the sense that prior knowledge of the signal model is not necessary in the quantizer design, only in the reconstruction. Thus, we demonstrate that it is possible to reduce the quantization error by incorporating side information on the acquired signal, such as sparse signal models or signal similarity with known signals. In doing so, we establish a relationship between quantization performance and the Kolmogorov entropy of the signal model.
Efficient Spoken Term Discovery Using Randomized Algorithms
"... Abstract—Spoken term discovery is the task of automatically identifying words and phrases in speech data by searching for long repeated acoustic patterns. Initial solutions relied on exhaustive dynamic time warpingbased searches across the entire similarity matrix, a method whose scalability is ult ..."
Abstract

Cited by 19 (13 self)
 Add to MetaCart
(Show Context)
Abstract—Spoken term discovery is the task of automatically identifying words and phrases in speech data by searching for long repeated acoustic patterns. Initial solutions relied on exhaustive dynamic time warpingbased searches across the entire similarity matrix, a method whose scalability is ultimately limited by the O(n 2) nature of the search space. Recent strategies have attempted to improve search efficiency by using either unsupervised or mismatchedlanguage acoustic models to reduce the complexity of the feature representation. Taking a completely different approach, this paper investigates the use of randomized algorithms that operate directly on the raw acoustic features to produce sparse approximate similarity matrices in O(n) space and O(n log n) time. We demonstrate these techniques facilitate spoken term discovery performance capable of outperforming a modelbased strategy in the zero resource setting. I.
Dimension reduction by random hyperplane tessellations
 Discrete & Computational Geometry
, 2011
"... Abstract. Given a subset K of the unit Euclidean sphere, we estimate the minimal number m = m(K) of hyperplanes that generate a uniform tessellation of K, in the sense that the fraction of the hyperplanes separating any pair x, y ∈ K is nearly proportional to the Euclidean distance between x and y. ..."
Abstract

Cited by 19 (3 self)
 Add to MetaCart
(Show Context)
Abstract. Given a subset K of the unit Euclidean sphere, we estimate the minimal number m = m(K) of hyperplanes that generate a uniform tessellation of K, in the sense that the fraction of the hyperplanes separating any pair x, y ∈ K is nearly proportional to the Euclidean distance between x and y. Random hyperplanes prove to be almost ideal for this problem; they achieve the almost optimal bound m = O(w(K)2) where w(K) is the Gaussian mean width of K. Using the map that sends x ∈ K to the sign vector with respect to the hyperplanes, we conclude that every bounded subset K of Rn embeds into the Hamming cube {−1, 1}m with a small distortion in the GromovHaussdorff metric. Since for many sets K one has m = m(K) n, this yields a new discrete mechanism of dimension reduction for sets in Euclidean spaces.
Sparse factor analysis for learning and content analytics
 J. OF MACHINE LEARNING RESEARCH
, 2014
"... We develop a new model and algorithms for machine learningbased learning analytics, which estimate a learner’s knowledge of the concepts underlying a domain, and content analytics, which estimate the relationships among a collection of questions and those concepts. Our model represents the probabil ..."
Abstract

Cited by 16 (10 self)
 Add to MetaCart
(Show Context)
We develop a new model and algorithms for machine learningbased learning analytics, which estimate a learner’s knowledge of the concepts underlying a domain, and content analytics, which estimate the relationships among a collection of questions and those concepts. Our model represents the probability that a learner provides the correct response to a question in terms of three factors: their understanding of a set of underlying concepts, the concepts involved in each question, and each question’s intrinsic difficulty. We estimate these factors given the graded responses to a collection of questions. The underlying estimation problem is illposed in general, especially when only a subset of the questions are answered. The key observation that enables a wellposed solution is the fact that typical educational domains of interest involve only a small number of key concepts. Leveraging this observation, we develop both a biconvex maximumlikelihoodbased solution and a Bayesian solution to the resulting SPARse Factor Analysis (SPARFA) problem. We also incorporate userdefined tags on questions to facilitate the interpretability of the estimated factors. Experiments with synthetic and realworld data demonstrate the efficacy of our approach. Finally, we make a connection between SPARFA and noisy, binaryvalued (1bit) dictionary learning that is of independent interest.
Secure Binary Embeddings for Privacy Preserving Nearest Neighbors
"... Abstract—We present a novel method to securely determine whether two signals are similar to each other, and apply it to approximate nearest neighbor clustering. The proposed method relies on a locality sensitive hashing scheme based on a secure binary embedding, computed using quantized random proje ..."
Abstract

Cited by 14 (7 self)
 Add to MetaCart
Abstract—We present a novel method to securely determine whether two signals are similar to each other, and apply it to approximate nearest neighbor clustering. The proposed method relies on a locality sensitive hashing scheme based on a secure binary embedding, computed using quantized random projections. Hashes extracted from the signals preserve information about the distance between the signals, provided this distance is small enough. If the distance between the signals is larger than a threshold, then no information about the distance is revealed. Theoretical and experimental justification is provided for this property. Further, when the randomized embedding parameters are unknown, then the mutual information between the hashes of any two signals decays to zero exponentially fast as a function of the ℓ2 distance between the signals. Taking advantage of this property, we suggest that these binary hashes can be used to perform privacypreserving nearest neighbor search with significantly lower complexity compared to protocols which use the actual signals. I.
Onebit measurements with adaptive thresholds
 Signal Processing Letters, IEEE
"... Abstract—We introduce a new method for adaptive onebit quantization of linear measurements and propose an algorithm for the recovery of signals based on generalized approximate message passing (GAMP). Our method exploits the prior statistical information on the signal for estimating the minimummea ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
(Show Context)
Abstract—We introduce a new method for adaptive onebit quantization of linear measurements and propose an algorithm for the recovery of signals based on generalized approximate message passing (GAMP). Our method exploits the prior statistical information on the signal for estimating the minimummeansquared error solution from onebit measurements. Our approach allows the onebit quantizer to use thresholds on the real line. Given the previous measurements, each new threshold is selected so as to partition the consistent region along its centroid computed by GAMP. We demonstrate that the proposed adaptivequantization scheme with GAMP reconstruction greatly improves the performance of signal and image recovery from onebit measurements. Index Terms—Analogtodigital conversion, approximate message passing, compressive sensing, onebit quantization. I.
Onebit compressed sensing with nongaussian measurements
, 2013
"... Abstract. In onebit compressed sensing, previous results state that sparse signals may be robustly recovered when the measurements are taken using Gaussian random vectors. In contrast to standard compressed sensing, these results are not extendable to natural nonGaussian distributions without furt ..."
Abstract

Cited by 13 (3 self)
 Add to MetaCart
(Show Context)
Abstract. In onebit compressed sensing, previous results state that sparse signals may be robustly recovered when the measurements are taken using Gaussian random vectors. In contrast to standard compressed sensing, these results are not extendable to natural nonGaussian distributions without further assumptions, as can be demonstrated by simple counterexamples involving extremely sparse signals. We show that approximately sparse signals that are not extremely sparse can be accurately reconstructed from singlebit measurements sampled according to a subgaussian distribution, and the reconstruction comes as the solution to a convex program.