In this paper, we provide theoretical results to show that compressed learning, learning directly in the compressed domain, is possible. In Particular, we provide tight bounds demonstrating that the linear kernel SVM’s classifier in the measurement domain, with high probability, has true accuracy close to the accuracy of the best linear threshold classifier in the data domain. We show that this is beneficial both from the compressed sensing and the machine learning points of view. Furthermore, we indicate that for a family of well-known compressed sensing matrices, compressed learning is universal, in the sense that learning and classification in the measurement domain works provided that the data are sparse in some, even unknown, basis. Moreover, we show that our results are also applicable to a family of smooth manifold-learning tasks. Finally, we support our claims with experimental results. 1

Abstract—We consider the problem of recovering a k-sparse signal (x) from hybrid (complex and real), noiseless compressive samples (y) using a mixture of complex-valued sparse and realvalued dense projections within a single matrix. The proposed Hybrid Compressed Sensing (HCS) employs the complex-sparse part of the projection matrix to divide the n-dimensional signal (x) into subsets. In turn, each subset of the signal (coefficients) is mapped onto a complex sample of the measurement vector (y). Under a worst-case scenario of such sparsity-induced mapping, when the number of complex sparse measurements is sufficiently large then this mapping leads to the isolation of a significant fraction of the k non-zero coefficients into different complex measurement samples from y. Using a simple property of complex numbers (namely complex phases) one can identify the isolated non-zeros of x. After reducing the effect of the identified non-zero coefficients from the compressive samples, we utilize the realvalued dense submatrix to form a full rank system of equations to recover the signal values in the remaining indices (that are not recovered by the sparse complex projection part). We show that the proposed hybrid approach can recover a k-sparse signal (with high probability) while requiring only m ≈ 3k 3 √ n/2k real measurements (where each complex sample is counted as two real measurements). We also derive expressions for the optimal mix of complex-sparse and real-dense rows within an HCS projection matrix. Further, in a practical range of sparsity ratio (k/n), the hybrid approach outperforms even the most complex compressed sensing frameworks (namely basis pursuit with dense Gaussian matrices). The theoretical complexity of HCS is less than the complexity of solving a full-rank system of m linear equations. In practice, the complexity can be lower than this bound. Index Terms—Compressed sensing, sparse projections, iterative decoding algorithms I.

Abstract—Sparse projections for compressed sensing have been receiving some attention recently. In this paper, we consider the problem of recovering a k-sparse signal (x) in an n-dimensional space from a limited number (m) of linear, noiseless compressive samples (y) using complex sparse projections. Our approach is based on constructing complex sparse projections using strategies rooted in combinatorial design and expander graphs. We are able to recover the non-zero coefficients of the k-sparse signal (x) iteratively using a low-complexity algorithm that is reminiscent of well-known iterative channel decoding methods. We show that the proposed framework is optimal in terms of sample requirements for signal recovery (m = O (k log(n/k))) and has a decoding complexity of O (m log(n/m)), which represents a tangible improvement over recent solvers. Moreover we prove that using the proposed complex-sparse framework, on average 2k < m ≤ 4k real measurements (where each complex sample is counted as two real measurements) suffice to recover a k-sparse signal perfectly. Index Terms—Compressed Sensing, sparse projections, channel decoding