Results 1  10
of
73
Wavelet shrinkage: asymptopia
 Journal of the Royal Statistical Society, Ser. B
, 1995
"... Considerable e ort has been directed recently to develop asymptotically minimax methods in problems of recovering in nitedimensional objects (curves, densities, spectral densities, images) from noisy data. A rich and complex body of work has evolved, with nearly or exactly minimax estimators bein ..."
Abstract

Cited by 297 (36 self)
 Add to MetaCart
(Show Context)
Considerable e ort has been directed recently to develop asymptotically minimax methods in problems of recovering in nitedimensional objects (curves, densities, spectral densities, images) from noisy data. A rich and complex body of work has evolved, with nearly or exactly minimax estimators being obtained for a variety of interesting problems. Unfortunately, the results have often not been translated into practice, for a variety of reasons { sometimes, similarity to known methods, sometimes, computational intractability, and sometimes, lack of spatial adaptivity. We discuss a method for curve estimation based on n noisy data; one translates the empirical wavelet coe cients towards the origin by an amount p p 2 log(n) = n. The method is di erent from methods in common use today, is computationally practical, and is spatially adaptive; thus it avoids a number of previous objections to minimax estimators. At the same time, the method is nearly minimax for a wide variety of loss functions { e.g. pointwise error, global error measured in L p norms, pointwise and global error in estimation of derivatives { and for a wide range of smoothness classes, including standard Holder classes, Sobolev classes, and Bounded Variation. This is amuch broader nearoptimality than anything previously proposed in the minimax literature. Finally, the theory underlying the method is interesting, as it exploits a correspondence between statistical questions and questions of optimal recovery and informationbased complexity.
Adapting to unknown sparsity by controlling the false discovery rate
, 2000
"... We attempt to recover a highdimensional vector observed in white noise, where the vector is known to be sparse, but the degree of sparsity is unknown. We consider three different ways of defining sparsity of a vector: using the fraction of nonzero terms; imposing powerlaw decay bounds on the order ..."
Abstract

Cited by 182 (23 self)
 Add to MetaCart
We attempt to recover a highdimensional vector observed in white noise, where the vector is known to be sparse, but the degree of sparsity is unknown. We consider three different ways of defining sparsity of a vector: using the fraction of nonzero terms; imposing powerlaw decay bounds on the ordered entries; and controlling the ℓp norm for p small. We obtain a procedure which is asymptotically minimax for ℓr loss, simultaneously throughout a range of such sparsity classes. The optimal procedure is a dataadaptive thresholding scheme, driven by control of the False Discovery Rate (FDR). FDR control is a recent innovation in simultaneous testing, in which one seeks to ensure that at most a certain fraction of the rejected null hypotheses will correspond to false rejections. In our treatment, the FDR control parameter q also plays a controlling role in asymptotic minimaxity. Our results say that letting q = qn → 0 with problem size n is sufficient for asymptotic minimaxity, while keeping fixed q>1/2prevents asymptotic minimaxity. To our knowledge, this relation between ideas in simultaneous inference and asymptotic decision theory is new. Our work provides a new perspective on a class of model selection rules which has been introduced recently by several authors. These new rules impose complexity penalization of the form 2·log ( potential model size / actual model size). We exhibit a close connection with FDRcontrolling procedures having q tending to 0; this connection strongly supports a conjecture of simultaneous asymptotic minimaxity for such model selection rules.
Efficient Online and Batch Learning using Forward Backward Splitting
"... We describe, analyze, and experiment with a framework for empirical loss minimization with regularization. Our algorithmic framework alternates between two phases. On each iteration we first perform an unconstrained gradient descent step. We then cast and solve an instantaneous optimization problem ..."
Abstract

Cited by 134 (1 self)
 Add to MetaCart
(Show Context)
We describe, analyze, and experiment with a framework for empirical loss minimization with regularization. Our algorithmic framework alternates between two phases. On each iteration we first perform an unconstrained gradient descent step. We then cast and solve an instantaneous optimization problem that trades off minimization of a regularization term while keeping close proximity to the result of the first phase. This view yields a simple yet effective algorithm that can be used for batch penalized risk minimization and online learning. Furthermore, the two phase approach enables sparse solutions when used in conjunction with regularization functions that promote sparsity, such as ℓ1. We derive concrete and very simple algorithms for minimization of loss functions with ℓ1, ℓ2, ℓ 2 2, and ℓ ∞ regularization. We also show how to construct efficient algorithms for mixednorm ℓ1/ℓq regularization. We further extend the algorithms and give efficient implementations for very highdimensional data with sparsity. We demonstrate the potential of the proposed framework in a series of experiments with synthetic and natural datasets.
Nonlinear Wavelet Methods for Recovery of Signals, Densities, and Spectra from Indirect and Noisy Data
 In Proceedings of Symposia in Applied Mathematics
, 1993
"... . We describe wavelet methods for recovery of objects from noisy and incomplete data. The common themes: (a) the new methods utilize nonlinear operations in the wavelet domain; (b) they accomplish tasks which are not possible by traditional linear/Fourier approaches to such problems. We attempt to i ..."
Abstract

Cited by 133 (5 self)
 Add to MetaCart
(Show Context)
. We describe wavelet methods for recovery of objects from noisy and incomplete data. The common themes: (a) the new methods utilize nonlinear operations in the wavelet domain; (b) they accomplish tasks which are not possible by traditional linear/Fourier approaches to such problems. We attempt to indicate the heuristic principles, theoretical foundations, and possible application areas for these methods. Areas covered: (1) Wavelet DeNoising. (2) Wavelet Approaches to Linear Inverse Problems. (4) Wavelet Packet DeNoising. (5) Segmented MultiResolutions. (6) Nonlinear Multiresolutions. 1. Introduction. With the rapid development of computerized scientific instruments comes a wide variety of interesting problems for data analysis and signal processing. In fields ranging from Extragalactic Astronomy to Molecular Spectroscopy to Medical Imaging to Computer Vision, one must recover a signal, curve, image, spectrum, or density from incomplete, indirect, and noisy data. What can wavelets ...
Wavelet Processes and Adaptive Estimation of the Evolutionary Wavelet Spectrum
, 1998
"... This article defines and studies a new class of nonstationary random processes constructed from discrete nondecimated wavelets which generalizes the Cramer (Fourier) representation of stationary time series. We define an evolutionary wavelet spectrum (EWS) which quantifies how process power va ..."
Abstract

Cited by 73 (28 self)
 Add to MetaCart
This article defines and studies a new class of nonstationary random processes constructed from discrete nondecimated wavelets which generalizes the Cramer (Fourier) representation of stationary time series. We define an evolutionary wavelet spectrum (EWS) which quantifies how process power varies locally over time and scale. We show how the EWS may be rigorously estimated by a smoothed wavelet periodogram and how both these quantities may be inverted to provide an estimable timelocalized autocovariance. We illustrate our theory with a pedagogical example based on discrete nondecimated Haar wavelets and also a real medical time series example.
Smooth Wavelet Decompositions with Blocky Coefficient Kernels
, 1993
"... We describe bases of smooth wavelets where the coefficients are obtained by integration against (finite combinations of) boxcar kernels rather than against traditional smooth wavelets. Bases of this type were first developed in work of Tchamitchian and of Cohen, Daubechies, and Feauveau. Our approac ..."
Abstract

Cited by 60 (12 self)
 Add to MetaCart
(Show Context)
We describe bases of smooth wavelets where the coefficients are obtained by integration against (finite combinations of) boxcar kernels rather than against traditional smooth wavelets. Bases of this type were first developed in work of Tchamitchian and of Cohen, Daubechies, and Feauveau. Our approach emphasizes the idea of averageinterpolation  synthesizing a smooth function on the line having prescribed boxcar averages  and the link between averageinterpolation and DubucDeslauriers interpolation. We also emphasize characterizations of smooth functions via their coefficients. We describe boundarycorrected expansions for the interval, which have a simple and revealing form. We use these results to reinterpret the empirical wavelet transform  i.e. finite, discrete wavelet transforms of data arising from boxcar integrators (e.g. CCD devices).
Generalized Cross Validation for wavelet thresholding
 Signal Processing
, 1995
"... Noisy data are often fitted using a smoothing parameter, controling the importance of two objectives that are opposite to a certain extent. One of these two is smoothness and the other is closeness to the input data. The optimal value of this paramater minimizes the error of the result (as compared ..."
Abstract

Cited by 48 (17 self)
 Add to MetaCart
(Show Context)
Noisy data are often fitted using a smoothing parameter, controling the importance of two objectives that are opposite to a certain extent. One of these two is smoothness and the other is closeness to the input data. The optimal value of this paramater minimizes the error of the result (as compared to the unknown, exact data), usually expressed in the L 2 norm. This optimum cannot be found exactly, simply because the exact data are unknown. In spline theory, the Generalized Cross Validation (GCV) technique has proven to be an effective (though rather slow) statistical way for estimating this optimum. On the other hand, wavelet theory is well suited for signal and image processing. This paper investigates the possibility of using GCV in a noise reduction algorithm, based on waveletthresholding, where the threshold can be seen as a kind of smoothing parameter. The GCV method thus allows choosing the (nearly) optimal threshold, without knowing the noise variance. Both an original theore...
HaarFisz estimation of evolutionary wavelet spectra
, 2008
"... We introduce a waveletbased model of local stationarity. This model enlarges the class of locally stationary wavelet processes and contains processes whose spectral density function may change very suddenly in time. A notion of timevarying wavelet spectrum is uniquely defined as a wavelettype tra ..."
Abstract

Cited by 28 (13 self)
 Add to MetaCart
We introduce a waveletbased model of local stationarity. This model enlarges the class of locally stationary wavelet processes and contains processes whose spectral density function may change very suddenly in time. A notion of timevarying wavelet spectrum is uniquely defined as a wavelettype transform of the autocovariance function with respect to socalled autocorrelation wavelets. This leads to a natural representation of the autocovariance which is localized on scales. We propose a pointwise adaptive estimator of the timevarying spectrum. The behavior of the estimator studied in homogeneous and inhomogeneous regions of the wavelet spectrum.
Estimating Covariances of Locally Stationary Processes: Rates of Convergence of Best Basis Methods
, 1998
"... Mallat, Papanicolaou and Zhang [MPZ98] recently proposed a method for approximating the covariance of a locally stationary process by a covariance which is diagonal in a specially constructed CoifmanMeyer basis of cosine packets. In this paper we extend this approach to estimating the covariance ..."
Abstract

Cited by 28 (10 self)
 Add to MetaCart
Mallat, Papanicolaou and Zhang [MPZ98] recently proposed a method for approximating the covariance of a locally stationary process by a covariance which is diagonal in a specially constructed CoifmanMeyer basis of cosine packets. In this paper we extend this approach to estimating the covariance from sampled data. Our method combines both wavelet shrinkage and cosinepacket bestbasis selection in a simple and natural way. The resulting algorithm is fast and automatic. The method has an interpretation as a nonlinear, adaptive form of anisotropic timefrequency smoothing. We introduce a new class of locally stationary processes which exhibits a form of inhomogeneous nonstationarity; our processes have covariances which typically change little from row to row, but might occasionally change abruptly. We study performance in an asymptotic setting involving triangular arrays of processes which are becoming increasingly stationary, and are able to prove rates of convergence results for our...