168 citations found. Retrieving documents...
Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105--.

 Home/Search   Document Not in Database   Summary   Related Articles   Check  

This paper is cited in the following contexts:

First 50 documents  Next 50

Probabilistic Models of Early Vision - Hoyer   (Correct)

....scheme adheres to the e#cient coding principle. The basic e#cient coding hypothesis states that sensory neurons should be adapted to transmit the maximum amount of information about the natural environment, given limited resources. This has also been called infomax, information maximization [13, 89, 103]. The limitations can be, for example, a fixed number of neurons and some fixed mean or maximum firing rates of these neurons. Alternatively, the hypothesis assumes that neurons minimize the resources needed to transmit a fixed amount of information. Using information theory, it is ....

R. Linsker, "Self-organization in a perceptual network," Computer, vol. 21, pp. 105-- 117, 1988.


Probabilistic Models of Early Vision - Hoyer (2002)   (Correct)

....scheme adheres to the e#cient coding principle. The basic e#cient coding hypothesis states that sensory neurons should be adapted to transmit the maximum amount of information about the natural environment, given limited resources. This has also been called infomax, information maximization [13, 89, 103]. The limitations can be, for example, a fixed number of neurons and some fixed mean or maximum firing rates of these neurons. Alternatively, the hypothesis assumes that neurons minimize the resources needed to transmit a fixed amount of information. Using information theory, it is ....

R. Linsker, "Self-organization in a perceptual network," Computer, vol. 21, pp. 105-- 117, 1988.


Source Separation as a By-Product of Regularization - Hochreiter, Schmidhuber (1999)   (1 citation)  (Correct)

....objective func tions (OFs) have been proposed to evaluate the quality of sensory codes. Most OFs focus on properties of the code components we refer to them as code componentoriented OFs, or COCOFs. Some COCOFs explicitly favor near factorial, mini mally redundant codes of the input data [2, 18, 23, 7, 24] while others favor local codes [22, 3, 16] Recently there has also been much work on COCOFs encouraging biologically plausible sparse distributed codes [20, 10, 25, 9, 6, 8, 12, 17] While COCOFs express desirable properties of the code itself they neglect the costs of constructing the code ....

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21:105- 117, 1988.


Noise and Information in Neural Codes - Schneidman   (Correct)

....to questions regarding the nature of the neural code [143, 22, 37] as we discuss below. It has also been used for quantifying the nature of jitter in the propagation of action potentials along the axon [6] the analysis of the nature of a synaptic connection function and synaptic learning rules [102, 103, 116, 29], the study of feature encoding [173] Especially interesting are the ideas of maximization of encoded information and e#cient information encoding as design principles of the nervous system [15, 11, 17, 102, 103] 3.3.1 Entropy, Relative Entropy and Mutual Information The entropy of a set of ....

.... the analysis of the nature of a synaptic connection function and synaptic learning rules [102, 103, 116, 29] the study of feature encoding [173] Especially interesting are the ideas of maximization of encoded information and e#cient information encoding as design principles of the nervous system [15, 11, 17, 102, 103]. 3.3.1 Entropy, Relative Entropy and Mutual Information The entropy of a set of responses of a system, measures how rich (or surprising ) the set of responses is. If a system is presented with a variety of stimuli and responds in the same way every time, or that only a few di#erent responses ....

[Article contains additional citation context not shown here]

R. Linsker. Self organization in a perceptual network. Computer, 21:105--117, 1988.


Information Theoretic Approaches to Neural Network Learning - Plumbley (1997)   (Correct)

....the edges and corners of an image, since they are the least predictable from their surroundings. This is consistent with the structural arrangement of simple cells now known to exist in the visual cortex. More recently, with the resurgence of the eld of neural networks, authors such as Linsker [12], Barlow and F oldi ak [6] Plumbley and Fallside [19] and Atick and Redlich [1] have continued the use of information theory in neural networks, with considerable success. Information theory has proved particularly useful in the development of unsupervised learning algorithms. Unlike supervised ....

....is more likely that early parts of the visual system should process all input information equally. Linsker therefore suggested his Infomax principle: that a perceptual system should attempt to organize itself to maximize the rate of information transmitted (in bits per second) through the system [12]. An alternative view introduced by Plumbley and Fallside [19] is to try to minimize the loss in information about some original signal as the sensory input is processed by the perceptual system or neural network. Although this approach is in many ways equivalent to Linsker s Infomax principle, ....

[Article contains additional citation context not shown here]

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21(3):105-117, March 1988.


Information Processing in Negative Feedback Neural Networks - Plumbley (1995)   (1 citation)  (Correct)

....almost as soon as the concept of mutual information was introduced by Shannon [1] For example, Barlow [2] suggested that neural networks with lateral inhibition connections could be used to achieve an economical description of perceptual information. More recently, work by authors such as Linsker [3] and Atick and Redlich [4] have applied information theory more directly to neural network learning. In this article we shall use this approach to develop an algorithm for a two stage neural network with negative feedback. 2. Principal subspace networks Suppose we have an N dimensional zero mean ....

....from x a to y a . If fact, for optimum information capacity it is sucient for the rows of W to span the same subspace as the rst M eigenvectors of xa : we call this principal subspace analysis. Any reversible rotation or scaling in the output y a space does not a ect the information capacity [3, 5]. A number of neural network algorithms have been developed to perform PCA or principal subspace analysis [6, 7, 5] many based on the Oja [8] principal component nding neuron. Often these use a learning algorithm of the form W = W (y a x a KW) 1) where W is a small update factor, and K ....

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21(3):105-117, March 1988.


Minimization of Information Loss through Neural Network Learning - Plumbley (1993)   (Correct)

....may hold in early processing stages of complex sensory systems such as the retina in higher mammals. 1 Introduction In recent years, a number of authors have used concepts from Information Theory to develop or explain neural network learning algorithms, particularly in sensory systems [1, 2, 3, 4, 5]. A neural network in a sensory system is thought of as part of a communication system, transmitting Shannon Information [6] about the outside world to further processing stages. The Centre for Neural Networks, Department of Mathematics, King s College London, Strand, London, WC2R 2LS, UK ....

....in a particular layer, and by costs, such as the average power used to transmit the information. In this article, we explore the concept of minimization of information loss (MIL) 2] as a target for neural network learning in this context. MIL is closely related to Linsker s Infomax principle [1]. By relating MIL to more familiar supervised and unsupervised learning procedures such as Error Back Propagation ( BackProp ) 7] and principal component analysis (PCA) 1] we show how it can be used as a lingua franca for learning in all stages of a neural network sensory system. 2 Mutual ....

[Article contains additional citation context not shown here]

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21(3):105--117, March 1988.


Information Theory and Neural Networks - Taylor, Plumbley (1993)   (3 citations)  (Correct)

....of an economical description of a scene from a very redundant initial representation. Barlow [9] suggested that lateral inhibition in the visual pathway may reduce the redundancy of an image, so information can be represented more eciently. More recently, Linsker with his Infomax principle [21, 22], Atick and Redlich [5, 6] and Plumbley and Fallside [30, 32] have continued with this approach with considerable success. There have also been important advances in data compression techniques associated with principal component analysis. The original work of Oja [23] has now been extended to ....

....learns to perform a principal component analysis of its input, but that principal component analysis itself su ers from an inconsistency problem when the scaling of the input components is not well de ned. In order to gain some insight to this problem, we shall apply Linsker s Infomax principle [21] to this situation. Consider a system with input X and output Y . Linsker s Infomax principle states that a network should adjust itself so that the information I(X; Y ) transmitted to its output Y about its input X should be maximised. This is equivalent to the information in the input X about ....

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21(3):105{ 117, Mar. 1988.


Approximating Optimal Information Transmission using Local.. - Plumbley (1993)   (2 citations)  (Correct)

....for optimizing MI with both input and output noise. This represents a better approximation than simply orthonormalising the principal subspace. 1 Introduction Recently, interest has been expressed in using information theory to investigate unsupervised learning in perceptual systems. Linsker [3] suggested that a perceptual system should adapt itself to maximize the mutual information (MI) between its input and its output. The type of network which optimizes information transmission depends considerably on the noise sources assumed to be present in the system. If the only noise source is ....

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21(3):105--117, Mar. 1988.


A Network which Performs Orthonormalized Principal Subspace.. - Plumbley (1994)   (Correct)

.... i i i i i1 P P P P Pq R Gamma Gamma Gamma Gamma Gamma i i i i i1 P P P P Pq j i j i y(t) W(t) Figure 1: Neural network implementation of a linear dimension reduction pre processor y(t) W(t)x(t) with N = 3 inputs and M = 2 outputs principle [9]. PCA is not the unique solution to either of these theoretical requirements, however. If the output to the network is transformed by any invertible linear (or even non linear) function, the maximization of information, and minimization of mean square reconstruction error, will still be ....

....as we take each successive output component. Also the PCA up to any M 1 components is part of the PCA up to some M 2 M 1 components, so we can form smaller PCA solutions simply by truncating the output vector from a larger PCA solution. From an information theoretic perspective, Linsker [9] observed that PCA maximizes the information extracted from a Gaussian input signal with uncorrelated equal variance additive Gaussian noise on this input signal. However, PCA as described above is not the only M Theta N linear transform which is optimal as far as maximization of information is ....

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21(3):105--117, Mar. 1988.


Adaptive Network for Optimal Linear Feature Extraction - Földiák   (Correct)

....exactly the desired inputoutput relationship will be. Selection of features can rely only on rcgularitics in the input data set. The quality of a set of features can be determined by information theoretic measures. Good features will reduce dimcnsionality with only a minimal loss of information [8, 10], Amen s linear methods Principal Component Analysis has such optimal properties. principal qomponent nnai,sis PCA is a statistical method for extracting features from high dimensional data distributions [1] It is a linear, orthogonal transformation (rotation) of a distribution into a ....

....observation of its output to maximize the decrease in our uncertainty about the input, i.e. it should maximize the mutual infor mation. Oja s algorithm reaches the maximum of mutual information set by PCA for the single unit case if the inputs contain uncorrciated noise of equal variance [8], and similarly for algorithms that yield the PeA subspace [9] It can be shown, that H(X) H(X I Y) H(Y) H(Y I X) where H(Y) is the entropy of the output, and H(Y I X) the conditional entropy of the output, represents the effect of noise in the input. The right hand side of the ....

R. Linsker, "Self-organization in a perceptual network", I., vol. 21, pp. 105-117, March 1988.


Information Theory and Neural Network Learning Algorithms - Plumbley (1992)   (2 citations)  (Correct)

....Not all psychologists agreed with this approach. Green and Courtis [4] for example, argued that the lack of an objective alphabet of symbols and transition probabilities meant that information theory could not be used. More recently, with the resurgence of the field of neural networks, Linsker [5], Barlow and Foldi ak [6] Plumbley and Fallside [7] and Atick and Redlich [8] have resurrected the use of information theory in neural networks, with several interesting results. This paper is organized as follows. Section 2 gives a brief introduction to information theory, Linsker s Infomax ....

....is more likely that early parts of the visual system should process all input information equally. Linsker therefore suggested his Infomax principle: that a perceptual system should attempt to organize itself to maximize the rate of information transmitted (in bits per second) through the system [5]. An alternative view introduced by Plumbley and Fallside [7] is to try to minimize the loss in information about some original signal Omega as the sensory input is processed by the perceptual system or neural network. Although this approach is in many ways equivalent to Linsker s Infomax ....

[Article contains additional citation context not shown here]

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21(3):105--117, Mar. 1988.


Entropy Optimization - Application To Blind Source Separation - Taleb, Jutten   (Correct)

....based upon MSE minimization is provided. Validation of this algorithm is given by an application to the Source Separation problem. 1 Introduction Unsupervised learning algorithms aim to find hidden structures and informative representations of large data sets. The infomax principle of Linsker [8], which is a fundamental principle of self organization, states that the transformation of a vector x observed on the input layer of a Neural Network (NN) to a vector y on the output, should be chosen in order to maximize the transinformation between input x and output y. The Exploratory ....

R. Linsker. Self-organization in a perceptual network. Computer, (21):105--117, 1988.


Dynamical properties of strongly interacting Markov chains - Ay, Wennekers (2001)   (Correct)

.... 23, 25] The hypothesis of strong interrelations is implicitly contained also in many conceptual approaches to the understanding of first principles for neural organization and learning, where information theory provides an appropriate framework for the formulation and analysis of such principles [15, 16, 6, 20]. A well known measure that quantifies relations of interacting units is a generalized version of the so called mutual information of two units: Consider N binary units 1; 2; N and a joint probability distribution p on the configuration set f0; 1g N . Then the Kullback Leibler ....

....case, it is always satisfied [7] In especially, Rodrigues, Huerta and Lopez [22] demonstrated the emergence of strong local interactions in an instructive model for neural networks. Experimental evidence for such a local optimization is also provided by Laughlin [14] cf. also [21] Linsker [15, 16] further established a connection between (local) Hebbian learning rules and the optimization of local interactions. Thus, we are encouraged to postulate the optimization of global interaction as a first principle for learning in neural networks. This principle has been proposed in the context of ....

[Article contains additional citation context not shown here]

Linsker, R. (1988) Self-organization in a perceptual network. IEEE Computer, 21, 105--117.


Dynamic Model of Visual Recognition Predicts Neural Response.. - Rao, al. (1995)   (31 citations)  (Correct)

.... Related Work on Cortical Modeling There has been considerable work in recent years on determining the computational nature of the cortex and the brain [Churchland and Sejnowski, 1992] This work includes models ranging from feedforward networks such as the hierarchical perceptual network of [Linsker, 1988] and the HBF networks of Poggio and collaborators [Poggio, 1990; Poggio et al. 1992 ] to bi directional flow control based models,such as Ullman s Counterstreams model [Ullman, 1994] Deacon s counter current diffusion model [ Deacon, 1989] and Van Essen et al. s Dynamic Routing Circuit ....

R. Linsker. Self-organization in a perceptual network. Computer, 21(3):105--117, 1988.


Efficient Encoding of Natural Time Varying Images Produces.. - Rao, Ballard (1997)   (1 citation)  (Correct)

....explanation of the center surround structure of retinal ganglion receptive fields in terms of whitening or decorrelation of outputs in response to natural images. Several Hebbian learning algorithms for decorrelation have also been proposed [Bienenstock et al. 1982; Williams, 1985; Barrow, 1987; Linsker, 1988; Oja, 1989; Sanger, 1989; Foldiak, 1990; Atick and Redlich, 1993 ] many of which perform Principal Component Analysis (PCA) Although the PCA of natural images produces lower order components that resemble oriented filters [Baddeley and Hancock, 1991; Hancock et al. 1992 ] the higher order ....

R. Linsker. Self-organization in a perceptual network. Computer, 21(3):105--117, 1988.


A Neural Network Model for Facial Affect Classification - Padgett   (Correct)

....It is developed using the principal components of a set of random blocks from the database. The same local feature regions of the face from the eigenfeature representation are projected onto this sub space. This type of representation is often associated with the early vision system [37]. Our motivation here is to develop a set of non specialized features (presumably like those found in the visual system) that are not specifically biased to the variation found in the areas of interest. Although the techniques for generating these features are the same used in finding the ....

R. Linsker. Self-organization in a perceptual network. Computer, pages 105--117, March 1988.


A Two-Layer Sparse Coding Model Learns Simple and Complex.. - Hyvärinen, Hoyer (2001)   (Correct)

....In this paper, we extend the sparse coding principle to model complex cell properties and topography. By topography, we mean the columnar or clustering organization of the cells. Several neural network models have been proposed for learning these properties (von der Malsburg, 1973; Kohonen, 1982; Linsker, 1988; Obermayer et al. 1990; Erwin et al. 1995; Miller, 1995; Kohonen, 1996; Swindale, 1996; Hyvarinen and Hoyer, 2000) but none has succesfully demonstrated emergence of all of them. Here we show that these properties emerge from a two layer sparse coding model that is fed natural image data as ....

Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21:105--117.


A Bayesian Unsupervised Learning Algorithm that Scales - Ghahramani, Hinton   (Correct)

.... components analysis (PCA) Unsupervised learning models based on Hebbian learning can generally be viewed as implementing variants of PCA (Oja, 1982) Models of this kind have been found to develop center surround and orientationselective properties similar to those of cells in the visual system (Linsker, 1988). In this paper, we first describe the need for models that go beyond factor analysis and mixtures of Gaussians. The goal of these models is to discover hierarchical distributed representations that are non linearly related to the perceptual data. We briefly review previous attempts at developing ....

Linsker, R. (1988). Self-organization in a perceptual network. IEEE Computer, 21:105--117.


Turbo Factor Analysis - Frey (1999)   (9 citations)  (Correct)

....describes principal component analysis, independent component analysis and factor analysis. constraint is given by E[xjz] Lz; 1) where E[xjz] R x xp(xjz)dx and each column of L is a component vector. L is an N K matrix with elements nk . Principal component analysis (Jolliffe 1986; Linsker 1988; Oja 1989) independent component analysis (Comon, Jutten and Herault 1991; Bell and Sejnowski 1995; Amari, Cichocki and Yang 1996) and factor analysis (Rubin and Thayer 1982; Everitt 1984) can be viewed as maximum likelihood estimation in models of this type, where we assume that the ....

Linsker, R. 1988. Self organization in a perceptual network. Computer, 21:105--128.


Scene Dependence of the Non-Gaussian Scaling Properties of .. - Nevado, Turiel, Parga (2000)   (1 citation)  (Correct)

....of the environment. In the case of visual scenes, a systematic study of this matter has began rather recently [2, 3, 4, 5, 6, 7] Although the relevance of the second order statistics has been pointed out some time ago [8, 9] and a gaussian distribution for images has very often been used [10, 11, 12, 13] to make predictions on the receptive elds of cells in V1 and in previous stages of the visual pathway, there are many reasons to believe that this statistics leaves aside a vast number of qualitatively important properties. An indication of this is that once the image is decorrelated (i.e. the ....

R. Linsker. Self-organization in a perceptual network. Computer, 21 105-117 (1988)


Statistical Language Processing based on Self-Organising Word.. - McMahon (1994)   (2 citations)  (Correct)

....5 . This opinion has recently been championed by Finch [53] it also bears similarities to the philosophy of science of Popper [127] Some connectionist researchers have described the functioning of their neural systems in terms of information maximisation, under certain constraints. Linsker [102] describes a multilayered neural net which uses Hebbian learning to self organise feature analysing cells. He states that the organising principle behind the changes in connection strength involves maximising the amount of information preserved in the signal as if moves through the layers. This is ....

Ralph Linsker. Self-organization in a perceptual network. I.E.E.E. Computer, 21(3):105 -- 117, 1988.


A New Clustering Algorithm For Segmentation Of Magnetic Resonance .. - Gokcay (2000)   (Correct)

....rule which prevents the divergence is proposed by Oja [Oja82] Oja85] adds a weight decay proportional to and results in the following update rule: 2. 39) There are many variations to this update rule, and the update rule is used very frequently in principal component analysis (PCA) Ama77] [Lin88] [San89] Oja83] Rub89] Pri90] There are linear and nonlinear versions [Oja91] Tay93] Kar94] Xu94] Pri90] Competitive networks [Gro76a] Gro76b] Lip89] Rum85] Mal73] Gro69] Pri90] can be used in clustering procedures [Dud73] Har75] Since in clustering there is no signal which to ....

Linsker, R., "Self-organization in a perceptual network," Computer, p105-117, March 1988.


Empirical Entropy Manipulation for Real-World Problems - Viola, Schraudolph, Sejnowski (1996)   (13 citations)  (Correct)

....beyond the scope of existing parametric entropy models. 1 Introduction Information theory is playing an increasing role in unsupervised learning and visual processing. For example, Linsker has used the concept of information maximization to produce theories of development in the visual cortex (Linsker, 1988). Becker and Hinton have used information theory to motivate algorithms for visual processing (Becker and Hinton, 1992) Bell and Sejnowski have used information maximization to solve the cocktail Author to whom correspondence should be addressed. Current address: M.I.T. 545 Technology Square, ....

Linsker, R. (1988). Self-organization in a perceptual network. IEEE Computer, pages 105--117.


Temporally Dependent Plasticity: An Information Theoretic.. - Chechik, Tishby (2000)   (Correct)

....we start by deriving a spike dependent learning rule from rst principles within a simple rate model and then compare it with the experimentally observed STDP. To derive our learning rule, we consider the principle of mutual information maximization. This idea, known as the Infomax principle [14], states that the goal of a neural network s learning procedure is to maximize the mutual information between its output and input. The current paper applies Infomax for a leaky integrator neuron with spiking inputs. The derivation suggests computational insights into the dependence of the ....

R. Linsker. Self-organization in a perceptual network. Computer, 21(3):105-117, 1988.


An Information Theoretic Method for Designing.. - Omid Jahromi And   (Correct)

..... 00 is still a multivariable nonlinear optimization problem in rotation angles 1 ; 2 ; N 1 . We, however, believe that it can be further re cast as a sequence of singlevariable optimizations. The idea here has a very close relation with the Linsker s principle of self organization [6]. We state our method as Algorithm 1: To minimize I(X ; UN ) for the structure shown in Fig. 4, minimize I(U i ; U i 1 ) for i = 1; N 1 in a successive manner. The above method basically says to minimize the information available at the last stage, one can minimize the amount of ....

R. Linsker. Self-organization in a perceptual network. Computer, 21(3):105--117, March 1988.


A Two-Layer Sparse Coding Model Learns Simple and Complex.. - Hyvärinen, Hoyer (2000)   (Correct)

....In this paper, we extend the sparse coding principle to model complex cell properties and topography. By topography, we mean the columnar or clustering organization of the cells. Several neural network models have been proposed for learning these properties (von der Malsburg, 1973; Kohonen, 1982; Linsker, 1988; Obermayer et al. 1990; Erwin et al. 1995; Miller, 1995; Kohonen, 1996; Swindale, 1996; Hyv#rinen and Hoyer, 2000) but none has succesfully demonstrated emergence of all of them. Here we show that these properties emerge from a two layer sparse coding model that is fed natural image data as ....

Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21:105117.


Mutual Information, Fisher Information, and Population Coding - Brunel, Nadal (2000)   (Correct)

.... A natural framework to study how neurons communicate, or transmit information, in the nervous system is information theory (see, e.g. Blahut, 1988; Cover Thomas, 1991) In recent years the use of information theory in neuroscience has motivated a large amount of work (e.g. Laughlin, 1981; Linsker, 1988; Barlow, Kaushal, Mitchison, 1989; Bialek, Rieke, de Ruyter van Steveninck, Warland, 1991; Van Hateren, 1992; Atick, 1992; Nadal Parga, 1994) A neurophysiologist often asks in an informal sense how much information the spike train of a single neuron, or of a population of neurons, provides ....

Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105-- 117.


Discovering Predictable Classifications - Schmidhuber, Prelinger (1992)   (11 citations)  (Correct)

....as an auto encoder. The auto encoder is trained to emit the reconstruction h p;l of T l s external input x p;l , thus forcing y p;l to tell something about x p;l . D l is defined as D l = 1 2 X p kh p;l Gamma x p;l k 2 : 7) 2. 3 INFOMAX Following Linsker s Infomax approach (Linsker, 1988), we might think of defining GammaD l explicitly as the mutual information between the inputs and the outputs of T l . We did not use Infomax methods in our experiments for the following reasons: 1 Simply maximizing the variance of the output units without obeying constraint (6) will not ....

....of T l to represent environmental properties that are statistically independent from environmental properties represented by the remaining output units. The procedure aims at generating binary factorial codes (Barlow et al. 1989) It is our preferred method, because (unlike the methods used by Linsker (1988), Becker and Hinton (1989) and Zemel and Hinton (1991) it has a potential for removing even non linear statistical dependencies 2 among the output units of some classifier. Let us define D l = Gamma 1 2 X i (s p;l i Gamma y p;l i ) 2 ; 8) where the s p;l i are the outputs of ....

Linsker, R. (1988). Self-organization in a perceptual network. IEEE Computer, 21:105--117.


Early-Visual Features as Determinants of Perceived Texture.. - Neveitt   (Correct)

....vision texture. 2.6 Whitening The response profiles of striate cells often overlap significantly, leading to highly correlated output responses. This has lead researchers to postulate mechanisms by which the cortex might remove redundant information in low level features. In a provocative paper [26], Linsker has evolved a locally connected, unsupervised neural network whose functional behavior matches that of a standard statistical technique 10 known as whitening. Whitening, or principal components analysis (PCA) is a method for decorrelating a set of points in a vector space using a ....

R. Linsker. Self-organization in a perceptual network. Computer Magazine, 21:105--117, 1988.


Blind Source Separation in the Presence of Weak Sources - Nadal, Korutcheva, Aires (2000)   (Correct)

....quantity s 3 2 c ln (14) is the smallest. We consider now the information that will be conveyed by a network processing the data, and ask for the contribution to this information by each source when the network performs BSS. 3. 2 Characterization from infomax The infomax criterion [8, 10] will allow us to get some more insight onto the link between the sources strengths and the amount of information that can be extracted from the data. We consider the information processing of the signal by a nonlinear network, and we are interested in computing the mutual information I(V; S) ....

R. Linsker 1988, Self-organization in a perceptual network. Computer, 21:105-17.


Information-Theoretic Learning Using Renyi's Quadratic Entropy - Jose Principe Dongxin (1999)   (7 citations)  (Correct)

....about the underlying probability density function (PDF) Thus the method can manipulate information as straightforwardly as the mean square error (MSE) criterion and potentially substitute it as the workhorse of learning theory. The innumerous applications of information theory to learning [11] [15] have postulated either the form of PDF, or restrictive the mappings to be linear or nonlinear but full rank, or both (see [17] for an extended discussion) Viola [14] was the first to propose a nonparametric method to estimate entropy from examples using Shannon s entropy and the law of large ....

Linsker R. "Self-organization in perceptual networks ", in Computer 21, 105-117, 1988.


Sparse Coding of Natural Images Produces Localized.. - Olshausen, Field (1995)   (13 citations)  (Correct)

....of the coefficients are as statistically independent as possible over an ensemble of natural images. Achieving statistical independence is desirable, because it makes explicit the structure in sensory signals [5, 6] One line of approach to this problem is based on principal components analysis [7, 8, 9], in which the goal is to find a set of mutually orthogonal basis functions that capture the directions of maximum variance in the data and for which the coefficients are pairwise decorrelated, ha i a j i = ha i i ha j i. The receptive fields that result from this process are not localized, ....

Linsker R (1988) Self-organization in a perceptual network. Computer, pp. 105117.


Linsker-type Hebbian Learning: A Qualitative Analysis On The.. - Feng, Pan (1997)   (3 citations)  Self-citation (Linsker)   (Correct)

.... the purpose of understanding the self organization mechanism of primary visual system, Linsker has proposed a multilayered unsupervised Hebbian learning network with random uncorrelated inputs and localized arborization of synapses between adjacent layers [20] The simulation results reported in [20, 21, 22] have shown that for appropriate parameter regimes, several structured connection patterns (e.g. center surround and oriented receptive fields) occur progressively as the Hebbian evolution of the weights is carried out layer by layer. In addition to exhibiting some of the major features of early ....

....perspective [27] these constraints provide the competition mechanism among fluctuations due to the limitation of resources. 3. The localized arborization of synapses between adjacent layers. Thus, this model applies to the formation of afferent receptive fields (aRFs) 1 [27] As observed in [20, 21, 22], and as rigorously shown in this paper, this factor plays a crucial role in the emergence of structured aRFs. The behavior of the Linsker s model is determined by the underlying nonlinear dynamics that are parameterized by a set of parameters originating from the Hebbian rule and the arbor ....

[Article contains additional citation context not shown here]

Linsker, R. (1988b). Self-organization in a perceptual network. Computer, 21(3), 105-117.


Redundancy Reduction with Information - Preserving Nonlinear Maps   (Correct)

No context found.

Linsker, R. (1988). Self-organization in a perceptual network. Computer, 21, 105--.


On maximization of the information divergence from an.. - Matus, Ay (2004)   (Correct)

No context found.

Linsker R. (1988) Self-organization in a perceptual network. IEEE Computer 21 105117.


Maximization of Information Divergences Related to Binary i.i.d.. - Matus (2004)   (Correct)

No context found.

R. Linsker (1988) Self-organization in a perceptual network. IEEE Computer 21 105--17.


Audio/Visual Independent Components - Smaragdis, Casey (2003)   (Correct)

No context found.

Linsker, R. (1988). Self-organization in a perceptual network. In Computer, 21.


2D Shape Analysis and Recognition Using Spatial Feature Extraction - Mount (1994)   (Correct)

No context found.

R. Linsker, "Self-organization in a perceptual network.," Computer, March 1988.


Appears in the IEEE Computer Society Conference on.. - Probabilistic..   (Correct)

No context found.

R. Linsker. Self-organization in a perceptual network. Computer, 21:105--117, 1988.


IEEE Trans. Pattern Analysis and Machine Intelligence.. - Evolutionary Pursuit And   (Correct)

No context found.

R. Linsker, "Self-organization in a perceptual network," Computer, vol. 21, pp. 105--117, 1988.


Adaptation and Unsupervised Learning - Peter Dayan Maneesh   (Correct)

No context found.

Linsker, R (1988) Self-organization in a perceptual network, Computer, 21, 105-128.


Feature Selection by Maximum Marginal Diversity - Vasconcelos (2003)   (1 citation)  (Correct)

No context found.

R. Linsker. Self-Organization in a Perceptual Network. IEEE Computer, 21(3):105--117, March 1988.


Source Separation as a - Product Of Regularization   (Correct)

No context found.

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21:105-- 117, 1988.


Neural networks versus Image Pyramids - Bischof, Kropatsch (1993)   (Correct)

No context found.

R. Linsker. Self-organization in a perceptual network. IEEE Computer, 21:105--117, 1988.


Vision and the Statistics of the Visual Environment - Simoncelli (2003)   (Correct)

No context found.

R Linsker. Self-organization in a perceptual network. Computer, 21:105--117, 1988.


Informative Features in Vision and Learning - Rudra (2002)   (Correct)

No context found.

R. Linsker, "Self-Organization in a Perceptual Network", Computer, pp. 105 117, March 1988,


Unsupervised Learning in Recurrent Neural Networks - Klapper-Rybicka..   (Correct)

No context found.

R. Linsker, \Self-organization in a perceptual network," IEEE Computer, vol. 21, pp. 105-117, 1988.


Unsupervised Learning in Recurrent Neural Networks - Klapper-Rybicka.. (2000)   (Correct)

No context found.

R. Linsker, \Self-organization in a perceptual network," IEEE Computer, vol. 21, pp. 105-117, 1988.


Observer-Participant Models of Neural Processing - Fry (1994)   (3 citations)  (Correct)

No context found.

Linsker, R. (1988). "Self-organization in a perceptual network," IEEE COMPUTER 105-117.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC