Field D.J. 1994. What is the goal of sensory coding? Neural Computation 6, 559-601  Home/Search   Document Not in Database   Summary   Related Articles   Check

....concrete, say that we are dealing with images of a fixed size of 256 by 256 pixels. This gives a total of 65536 = 256 pixels in an image. Each image can then be considered as a point, call it x, in a 65536 dimensional state space, each axis of which specifies the intensity value of one pixel [37]. Conversely, each point in the state space specifies one particular image. This state space concept is illustrated in figure 4.1. Next, consider taking an enormous set of images, and plotting each as the corresponding point in our state space. Of course, plotting a 65536 dimensional space is ....

....builds an internal model of the world, utilizing the redundancy in the sensory input. Independent component analysis based on a latent variable model 7. 1 Sparse coding In a seminal paper published in 1996, Olshausen and Field [110] described how a simple neural network performing sparse coding [9, 37, 39, 148] learned features that were qualitatively very similar to the receptive fields of V1 simple cells. This was significant because it was the first study to show how all the basic spatial properties of simple cell classical receptive fields (localization in space and in spatial frequency, and ....

D. J. Field, "What is the goal of sensory coding?," Neural Computation, vol. 6, pp. 559--601, 1994.

....concrete, say that we are dealing with images of a fixed size of 256 by 256 pixels. This gives a total of 65536 = 256 pixels in an image. Each image can then be considered as a point, call it x, in a 65536 dimensional state space, each axis of which specifies the intensity value of one pixel [37]. Conversely, each point in the state space specifies one particular image. This state space concept is illustrated in figure 4.1. Next, consider taking an enormous set of images, and plotting each as the corresponding point in our state space. Of course, plotting a 65536 dimensional space is ....

....builds an internal model of the world, utilizing the redundancy in the sensory input. Independent component analysis based on a latent variable model 7. 1 Sparse coding In a seminal paper published in 1996, Olshausen and Field [110] described how a simple neural network performing sparse coding [9, 37, 39, 148] learned features that were qualitatively very similar to the receptive fields of V1 simple cells. This was significant because it was the first study to show how all the basic spatial properties of simple cell classical receptive fields (localization in space and in spatial frequency, and ....

D. J. Field, "What is the goal of sensory coding?," Neural Computation, vol. 6, pp. 559--601, 1994.

....on rst and second order statistics (the covariance matrix) In PCA, the rows of W are in fact the eigenvectors of the covariance matrix of the data. Second order statistics capture the amplitude spectrum of images but not their phase spectrum. The high order statistics capture the phase spectrum [19, 12]. For a given sample of natural images we can scramble their phase spectrum while maintaining their power spectrum. This will dramatically alter the appearance of the images but will not change their second order statistics. The phase spectrum, not the power spectrum, contains the structural ....

....of Gaussian sources implicit in PCA makes it inadequate when the true sources are non Gaussian. In particular it has been empirically observed that many natural signals, including speech, natural images, and EEG are better described as linear combinations of sources with long tail distributions [19, 11]. These sources are called high kurtosis , sparse , or super Gaussian sources. Logistic random variables are a special case of sparse source models. When sparse source models are appropriate, ICA has the following potential PCA Projection Second IC First IC First PC Second PC Third PC and ....

[Article contains additional citation context not shown here]

D.J. Field. What is the goal of sensory coding? Neural Computation, 6:559-601, 1994. 28

....componentoriented OFs, or COCOFs. Some COCOFs explicitly favor near factorial, mini mally redundant codes of the input data [2, 18, 23, 7, 24] while others favor local codes [22, 3, 16] Recently there has also been much work on COCOFs encouraging biologically plausible sparse distributed codes [20, 10, 25, 9, 6, 8, 12, 17]. While COCOFs express desirable properties of the code itself they neglect the costs of constructing the code from the data. e.g. coding input data without redun Advances in Neural Information Processing Systems 10, MIT Press, Cambridge MA, 1999. dancy may be very expensive in terms of ....

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559- 601, 1994.

....coding, edge detection, fractal, multifractal EDICS: 1 STIL: Still Image Coding. I. INTRODUCTION E DGE dectection is a common feature of the mammals visual neural system[2] 3] It has been proposed that edge detectors could be used to provide efficient coding algorithms [4], and in fact maximization of the information transfer lead to orientational edge dectecting filters [5] However, providing a reasonable, non conventional definition of edge is more controversial [6] A different strategy to produce efficient coding can be that of the statistical analysis of ....

D. J. Field, "What is the goal of sensory coding?," Neural Computation, vol. 6, pp. 559--601, 1994.

....filters in the retina. II A Image of center ON contrast detection filter. 2(b) Radial cut of the DOG filter. The vertical striped lines correspond to the variance of the narrower Gaussian used to generate the DOG filter. appropriate filters to detect contrasts in the image. As in [1] and from [12], neurons are defined according to their position and scale as dilated, translated and sampled Mexican Hat (or The # factor [11] of every image was controlled to assure the balance of luminance and mimic the analogical response of photo receptors to luminosity, i.e. to light intensity. PhRs ....

D. J. Field, "What is the goal of sensory coding?" Neural Computation, vol. 6, no. 4, pp. 559--601, 1994.

....we suggest that the kurtosis should be avoided as a sparseness measure and recommend tanh functions for measuring noisy sparseness. 1. INTRODUCTION In image analysis and vision research, sparseness has been demonstrated to be a powerful concept in finding meaningful representations of data [4, 11, 12, 3, 6, 7, 17, 15, 10]. The concept of sparseness or sparsity is also used in speech and music analysis [9, 2] in the statistical modeling of natural languages [16] and in various other applications. Despite the popularity of the sparse ideology , sparseness is not unambiguously defined. The simplest definition of ....

D. J. Field. What is the goal of sensory coding. Neural Computation, 6(4):559--601, 1994.

....world could be varying the most along the dimensions of ocular dominance, orientation preference and spatial frequency, and if so, the self organized RFs will represent these dimensions. During visual processing, the cortex projects incoming visual inputs onto these dimensions. As shown by Field [8], such a projection produces a sparse coding of the input. Projecting onto the dimensions of maximum variance also achieves minimal distortion and minimal spurious conjunctions of features. In sum, the RF LISSOM model predicts that the cortex performs two different computations during sensory ....

D. J. Field. What is the goal of sensory coding? Neuval Computation, 6:559 601, 1994.

....transforms of images onto local linear bases, i.e. linear filters, through convolution and then analyze filter responses. Examples include wavelets, steering filters, Gabors etc. An important principle for deriving the filters in the past has been to maximize the sparseness of filter responses [4, 12]. However, the resulting filters may not provide optimal performances in specific applications. The main goal of this paper is to present a family of algorithms for finding linear representations of images that are optimal for specific tasks and specific datasets. Our search for optimal linear ....

....search for linear filters that are optimal under some pre specified criteria. Studies of natural image statistics have shown that sparse filters of natural images share the important characteristics of receptive fields found in the early stages of the mammalian visual pathway [12] It was argued [4] that these filters should be important for visual recognition as it is the central task of the visual system. While sparseness has been a common consideration in deriving filters, few studies have attempted to relate filters to recognition performance. We attempt to make this connection in this ....

D. J. Field. What is the goal of sensory coding? Neural Computation, 6(4):559--601, 1994.

....with whitened images. 1. INTRODUCTION Properties of visual receptive fields in mammalian primary visual cortex has been extensively studied (e.g. 3, 4, 8] From these works, most of the theories of sensory coding have proposed models to effective internal representation by redundancy reduction [1, 5, 10, 15]. Among systems that produce such effects, Independent Components Analysis provides Gabor like detectors, similar to simple cells of the primary visual cortex, whose activities are statistically independent. The resulting coding has two main properties (sparseness and dispersal) describing how ....

Field D.J. (1994). What is the Goal of Sensory Coding ?, Neural Computation, vol. 6, pp. 559-601.

....by suggesting that these results be compared with other widely used auditory representations such as short term Fourier transforms, wavelet transforms, and physiologically derived models based on the auditory filterbank. 1. REDUNDANCY REDUCTION AS A GOAL OF PERCEPTION It has been suggested [2, 1, 7] that the processing of sensory data in biological perceptual systems is best understood in the language of information theory. The wealth of structure present in natural phenomena means that sensory signals are highly redundant; characterising this structure in order to develop efficient, ....

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

....or maximizing sparseness or independence, while preserving information. While such approaches had some considerable successes in explaining aspects of codes in the early processing stages of the mammalian visual system, e.g. Attick and Redlich, 1993, Barlow, 1989, Bell and Sejnowski, 1997, Field, 1994] it may be doubted that these ideas alone are sufficient to explain the brain s higher level visual representations. Two putative reasons for the insufficiency of these approaches lie in the active and purposive nature of biological vision [Aloimonos et al. 1988, Ballard, 1991, Aloimonos, ....

.... learning, e.g. Foldiak, 1991] Other work on unsupervised learning on natural images has provided models for the formation of receptive fields akin to those found in early processing stages of the mammalian visual system [Attick and Redlich, 1993, Barlow, 1989, Bell and Sejnowski, 1997, Field, 1994] We argue that for explaining visual representations at higher stages these approaches are too limited because they neglect the purposive and active nature of vision. An active vision system will shape the statistics of images that it sees in a purposeful manner and this can have a dramatic ....

Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6:559--601.

....weights. They experimented with several types of rectification constraint, finding that on the well known bars problem, an exponential nonlinearity on the output performed better than simple semilinear non negative constraint or a sigmoid, suggesting that this encourages a sparse representation [6]. Hoyer and Hyvarinen [7] explicitly combined a nonnegative constraint with a sparseness property, requiring the sources to have probability densities highly peaked at zero and with heavy tails. When provided with complex cell responses to natural images patches, their network learned basis ....

D. J. Field, "What is the goal of sensory coding?," Neural Computation, vol. 6, pp. 559--601, 1994.

....that is necessary to analyse images efficiently by only considering second order statistics. Therefore a compact coding such as the well known principal components analysis does not suffice. Many arguments have been put forward that imply a more relevant code for natural images is a sparse coding [12], i.e. a coding which produces outputs with high kurtosis. 5.1. The EPP algorithm and sparse coding As the EPP algorithm searches for codes with high kurtosis, it is suitable for coding natural images. Due to the large scale nature of the experiments, the performance may be improved by using a ....

....in section 5.1. We randomly sampled the pre processed images by taking 12 by 12 pixel patches, which were used as inputs to the network. Figure 2 shows a sample of 10 weightvectors, after the network was fully trained. These results are similar to those obtained by other sparse coding networks [12] and have been related to the receptive fields of simple cells in the Striate cortex. 5.2. Contextual Guidance The idea of contextual guidance in early visual processing can be simulated using the ECA network. For this experiment we chose 9 natural images, which we pre processed as decribed in ....

David J. Field, "What is the goal of sensory coding," Neural Computation, vol. 6, pp. 559--601, 1994.

....base probabilities of coe#cient activation. This sparse coding constraint encourages a model to use relatively few basis functions to represent any specific input signal. If the data has certain statistical properties (it is sparse ) this kind of coding leads to approximate redundancy reduction [17]. Sparse encoding within neural networks has previously been shown to create more biologically plausible receptive fields (Olshausen Field) 3.1 Evolutionary Algorithm for proposed sparse coding Some research has been done in applying genetic algorithms (GA) to the blind source separation ....

Field, D. J.: What is the goal of sensory coding? Neural Computation 6 (1994) 559--601

....of the j th basis function and S (a j ) is a sparseness term given by #log(1 (a j #) where # and # are scaling factors. This sparse coding constraint encourages the model to use relatively few basis functions to represent the input signal. This leads to approximate redundancy reduction [15]. Thus, the maximization of the log probability in (2) becomes: a = arg min a # #N 2 x Ma S(a j ) 5) 3 Evolutionary algorithm for proposed sparse coding From the model presented in Sect. 2, we derive a simple and robust learning algorithm by maximizing the data likelihood over ....

Field, D. J.: What is the Goal of Sensory Coding? Neural Computation 6 (1994) 559--601

....are referred to as subgaussian and positive kurtotic supergaussian or sparse variables. In our problems we can use kurtosis as an additional statistic for prior information on the distribution of the independent components. A close relationship between sparsity and ICA has been pointed out [8 10, 7]. In our par ticular problem, as in many others, a very high sparsity is observed in the independent components. Classification can be interpreted in terms of sparsity in the following way. If an independent feature corresponds to an object belonging to a certain class, then a sparse ....

Field, D.: What is the goal of sensory coding? Neural Computation 6 (1994) 559-601

.... have previously shown that nonnegativity is a useful constraint for matrix factorization that can learn a parts representation of the data [4, 5] The nonnegative basis vectors that are learned are used in distributed, yet still sparse combinations to generate expressiveness in the reconstructions [6, 7]. In this submission, we analyze in detail two numerical algorithms for learning the optimal nonnegative factors from data. 2 Non negative matrix factorization We formally consider algorithms for solving the following problem: Non negative matrix factorization (NMF) Given a non negative matrix ....

Field, DJ (1994). What is the goal of sensory coding? Neural Comput. 6, 559--601.

....a filter with a narrower radial passband. Figure 5.8 shows the frequency response of one basis function of the QMF pyramid before and after Gaussian blur. Field has observed that the kurtosis of filter outputs for natural images tends to peak for a radial filter bandwidth around one octave [32]. The QMF has a bandwidth of one octave, and we indeed find that narrowing that bandwidth through Gaussian filtering We use both f and # to denote the modulus of frequency. The units of f are cycles per unit distance while those of # are radians per unit distance, so # =2#f. The e#ects of a ....

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

....weights. They experimented with several types of rectification constraint, finding that on the well known bars problem, an exponential nonlinearity on the output performed better than simple semilinear non negative constraint or a sigmoid, suggesting that this encourages a sparse representation [6]. Hoyer and Hyvarinen [7] explicitly combined a nonnegative constraint with a sparseness property, requiring the sources to have probability densities highly peaked at zero and with heavy tails. When provided with complex cell responses to natural images patches, their network learned basis ....

D. J. Field, "What is the goal of sensory coding?," Neural Computation, vol. 6, pp. 559--601, 1994.

....by suggesting that these results be compared with other widely used auditory representations such as short term Fourier transforms, wavelet transforms, and physiologically derived models based on the auditory filter bank. 1. REDUNDANCY REDUCTION AS A GOAL OF PERCEPTION It has been suggested [2, 1,7] that the processing of sensory data in biological perceptual systems is best understood in the language of information theory. The wealth of structure present in natural phenomena means that sensory signals are highly redundant; characterising this structure in order to develop efficient, ....

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559-601, 1994.

....and an optimisation algorithm for dealing with sharply peaked priors. i Unsupervised Learning and Perception For some time now, unsupervised learning has appeared to be a most effective framework for understanding at least the early stages of sensory processing (see, for example [Barlow, 1989, Field, 1994] Perceptual processes are thought to be driven by a need to find representations of sensory signals that are both efficient and useful, as quantified by the related measures of low redundancy, high information content, and statistical independence [Atick, 1992] This sta tistical independence ....

Field, D. J. (1994). What is the goal of sensory coding? Neural Compu- tation, 6:559-601. 27

....(PCA) is used for computing global features of a signal, whereby a statistically stationary environment is assumed: all features should have the same probability of occurrence in signal space. However, in practice this often leads to inferior results when used for feature extraction purposes [2]. Furthermore, it cannot produce localized features [2] which means that there is a loss of accuracy in terms of the exact position of the feature. In an attempt to overcome the first problem, Dony and Haykin [1] combined the principles of PCA with competitive learning in order make the locally ....

....signal, whereby a statistically stationary environment is assumed: all features should have the same probability of occurrence in signal space. However, in practice this often leads to inferior results when used for feature extraction purposes [2] Furthermore, it cannot produce localized features [2], which means that there is a loss of accuracy in terms of the exact position of the feature. In an attempt to overcome the first problem, Dony and Haykin [1] combined the principles of PCA with competitive learning in order make the locally extracted features better match the local statistical ....

D.J. Field, "What Is the Goal of Sensory Coding?", Neural Computation, Vol. 6, pp. 559-601, 1994.

....requirements could be relaxed in favour of a bounded minimum requirement. We note that the proof relies on the sources being what we call well grounded, i.e. that they have a non vanishing pdf down to zero. There is much current interest in the analysis of observations generated by sparse sources [3], 4] 5] Sparse sources have large concentration of probability around zero, representing a high probability of being o . We would expect that a learning algorithm developed from the principles outlined in this letter would work particularly well for sources with this type of distribution. ....

D. J. Field, \What is the goal of sensory coding?," Neural Computation, vol. 6, pp. 559-601, 1994.

No context found.

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

No context found.

Field D.J. 1994. What is the goal of sensory coding? Neural Computation 6, 559-601

No context found.

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

No context found.

Field, D. J.: 1994, `What is the goal of sensory coding'. Neural Computation 6, 559#601.

No context found.

Field, D. J. (1994), `What is the goal of sensory coding', Neural Computation pp. 559-- 601.

No context found.

D. Field, "What Is the Goal of Sensory Coding," Neural Comp. 6, pp. 559-601, 1994.

No context found.

D. Field, "What is the goal of sensory coding," Neural Computation, vol. 6, pp. 559--601, 1994.

No context found.

D. Field, "What Is the Goal of Sensory Coding", Neural Comp. 6, p.559-601, 1994.

No context found.

D. Field, "What is the goal of sensory coding," Neural Computation, vol. 6, pp. 559--601, 1994.

No context found.

D. Field, "What is the goal of sensory coding," Neural Computation, vol. 6, pp. 559--601, 1994.

No context found.

D.J. Field. what is the goal of sensory coding? In Neural Computation 6 pages 559-601, 1994.

No context found.

D. Field, "What Is the Goal of Sensory Coding," Neural Comp. 6, pp. 559-601, 1994.

No context found.

D.J. Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

No context found.

D. J. Field, "What is the goal of sensory coding?" Neural Comput. 6, 559--601 (1994). 1250 J. Opt. Soc. Am. A / Vol. 20, No. 7 / July 2003 Hyva rinen et al.

No context found.

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

No context found.

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559-- 601, 1994.

No context found.

D. J. Field, "What is the goal of sensory coding?" Neural Comput. 6, 559--601 (1994).

No context found.

D.J. Field, "What Is the Goal of Sensory Coding?" Neural Computation, vol 6, pp. 559-601, 1994.

No context found.

D. J. Field, What is the goal of sensory coding?, Neural Computation 6 (4) (1994) 559--601.

No context found.

D. J. Field, "What is the goal of sensory coding?" Neural Comput. 6, 559--601 (1994).

No context found.

D.J. Field. What is the goal of sensory coding? Neural Computation, 6:559-601, 1994.

No context found.

D. J. Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

No context found.

D. Field, "What Is the Goal of Sensory Coding", Neural Comp. 6, p.559-601, 1994.

No context found.

D J Field. What is the goal of sensory coding? Neural Computation, 6:559--601, 1994.

No context found.

D. J. Field. What is the goal of sensory coding? Neural Computation, 6(4):559--601, July 1994.

No context found.

Field, D. "What is the goal of sensory coding?", Neural Computation. 6, 559-601, 1994.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC