G. E. Hinton, J. L. McClelland, and D. E. Rumelhart. Distributed representations. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1: Foundations, pages 77--109. MIT Press, Cambridge, MA, 1986.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....In general, it is difficult to tell exactly what systems with distributed knowledge representations know or don t know. Such systems cannot explain what they know, nor can a person look at their structures and tell whether they are in fact complete and robust or not, except in very simple cases [12] The only way to test such systems is by giving them examples and judging on the basis of their performance whether they are suitable or not. This problem is a quite serious one for systems that are designed to be fault tolerant. A fault tolerant system, for instance, might usually work quite ....

Hinton, G.E., McClelland, J.L., and Rumelhart, D.E. "Distributed Representations," Parallel Distributed Processing. Cambridge, MA: MIT Press, 1986.

....indicates corresponding author. # Sung Chan Jun, Barak A. Pearlmutter and Guido Nolte are with the Department of Computer Science, University of New Mexico, Albuquerque, NM 87131, USA (Email: junsc,bap,nolte cs.unm.edu) 1 The term distributed representation is standard in neural networks [4]. We experimented with a dataset containing as targets both the location and moment of each dipole, and despite the increased generalization expected for We made two datasets, one for training and the other for testing. Dipoles were drawn uniformly from truncated spherical regions [3, Figure ....

Geoffrey E. Hinton, James L. McClelland, and David E. Rumelhart. Distributed representations. In D. E. Rumelhart, J. L. McClelland, and the PDP research group., editors, Parallel distributed processing: Explorations in the microstructure of cognition, Volume 1: Foundations. MIT Press, 1986.

....far from biology, mainly for mathematical simplicity reasons [16] 18] Another item considered fundamental in a biologically based model is the representation adopted. It is required to be distributed, in a sense that one concept is represented along many units of the connectionist architecture [9] [14] while localist representations lack semantic distinctiveness [5] 17] Natural language processing systems that use distributed representations have shown good performance [11] 21] 22] 19] Here, a connectionist NLP system called Bio R is presented to account for thematic role ....

....as main characteristics, distributed representation, inhibitory competition, bidirectional activation propagation, and error driven task learning. 3.1. Distributed representation Several are the advantages of the distributed representation concerning connectionism. According to Hinton and others [9], the connections between a set of units are capable of supporting a large number of different patterns, thus implying in a considerable reduction of the network size. And, regarding cognition, the strengths and weaknesses give rise to some powerful and unexpected emergent properties, like ....

[Article contains additional citation context not shown here]

G. E. Hinton, J. L. McClelland, and D. E. Rumelhart, "Distributed Representations", in D. E. Rumelhart and J. L. McClelland (Eds.), Parallel Distributed Processing -- Explorations in the Microstructure of Cognition, Volume 1 -- Foundations. A Bradford Book, MIT Press, 1986, pp. 77109.

....incorporating different reasoning modes. One question is how to model the subconceptual, holistic similarity matching process within such a two level framework. As suggested in Dreyfus and Dreyfus [13] and Smolensky [43] distributed representation seems to be the best choice we have. Hinton [22] posited a similar hypothesis. The above conjectures lead naturally to the CONSYDERR architecture [45, 50, 52] Briefly, CONSY DERR consists of two levels: CL and CD. CL is a connectionist network with localist representation, or roughly reasoning at the conceptual level (Smolensky [43] Rules ....

G. Hinton et al., Distributed representations, in: D. Rumelhart et al., Eds., Parallel Distributcd Processing I (MIT Press, Cambridge, MA, 1986).

....in the next section. 3.3 Complexity in Memory Coding This section pursues binding representations in the brain, based on assumptions enumerated in the previous section. The argument is not based on any presumption of a specific mechanism or coding in the brain, such as distributed representation [19] and population coding [14] or the push down stack[44] Rather, we discuss the conditions, which must be satisfied by any mechanism 36 that performs language understanding. 3.3.1 Requirement of Additiveness Binding representations can be classified into additive ones and multiplicative ones. ....

....of a scene with two colored objects becomes a superposition of the representations of two colors and two objects. Then we cannot distinguish which color is bound to which object in the output; this is called a feature binding problem. It should be noted that a distributed representation [19] also su#ers from this problem. We assume that semantic information can be extracted from the distributed representation; otherwise it cannot be regarded as a semantic representation. Then, the extraction mechanism is either multiplicative or additive, depending on the utilization of ....

[Article contains additional citation context not shown here]

G. E. Hinton, James L. McClelland, and David E. Rumelhart. Distributed representations. In D. E. Rumelhart, J. L. McClelland, and PDP Research Group, editors, Foundation, number 1 in Parallel Distributed Processing: 132 Explorations in the Microstructure of Cognition, chapter 3, pages 77--109. MIT Press, Cambridge, MA, 1986. (cited in pages 36, 38, 52)

....when they identified that a network trained to map text to phonemes in fact had clustered similar sounding words close to each other in the hidden activation space. 6 van Gelder has internal hidden representations in mind, therefore the grouping emerges as a result of learning. Others (Hinton et al. 1986; Goschke and Koppelberg, 1991; van Gelder, 1991b; Chalmers, 1992; Clark, 1993) have argued that internal hidden states of connectionist networks may be construed as representations of the input constituents. Irrespective of whether the groupings emerge or are otherwise constructed, we believe ....

Hinton, G. E., McClelland, J. L., and Rumelhart, D. E. (1986). Distributed representations. In Rumelhart, D. E., McClelland, J., and the PDP Research Group, editors, Parallel Distributed Processing: Volume 1. MIT Press.

....are always identical, and uncorrelated. A recent introduction to Fisher information can be found in Kay (1993) 2 Scaling Rule for Tuning Width The problem of how coding accuracy depends on the tuning width of neurons and dimensionality of the space being represented was #rst studied by Hinton, McClelland, and Rumelhart (1986) and later by Baldi and Heiligenberg (1988) Snippe and Koenderink (1992) Zohary (1992) and Zhang et al. 1998) All of these earlier results involved speci#c assumptions on the tuning functions, the noise, and the measure of coding accuracy. Here we consider the general case using Fisher ....

....of function . Equation 2.3 gives the complete dependence of J on tuning width and number density . The factor D2 is consistent with the speci#c examples considered by Snippe and Koenderink (1992) and Zhang et al. 1998) but the exponent is off by one from the noiseless model considered by Hinton et al. 1986). More generally, when different neuron groups have different tuning widths and peak #ring rates F, we have J D D D2 K .F; D E ; 2.4) where the average is over neuron groups, and is the number density including all groups so that J is still proportional to the total number of ....

Hinton, G. E., McClelland, J. L., & Rumelhart, D. E. (1986). Distributed representations. In D. E. Rumelhart & J. L. McClelland (Eds.), Parallel distributed processing (Vol. 1, pp. 77--109). Cambridge, MA: MIT Press.

.... emphasizing the importance of single neurons for perception and motor control (Lettvin et al. 1959; Barlow, 1972) On the other hand, theoretical arguments suggest that in most cases broadly tuned units and distributed information processing are better suited for accurate representations (Hinton et al. 1986; Georgopoulos et al. 1986; Baldi and Heiligenberg, 1988; Snippe and Koenderink, 1992; Seung and Sompolinsky, 1993; Salinas and Abbott, 1994; Snippe, 1996; Eurich and Schwegler, 1997; Zhang et al. 1998; Zhang and Sejnowski, 1999) A useful measure of the information content of a 2 set of spike ....

Hinton, G. E., McClelland, J. L., and Rumelhart, D. E. (1986). Distributed representations. In Rumelhart, D. E. and McClelland, J. L., editors, Parallel Distributed Processing, volume 1, pages 77--109. MIT Press, Cambridge, MA.

....1. On the other hand, if the meaning of word1 has nothing in common with that of word2. then the value will be 0. The distributed nature of these representations arises because the concept meaning is captured by the set of attribute,value pairs as a whole rather than by one piece of information (Hinton, McClelland and Rumelhart, 1986). The most useful ramification of this from our perspective is that the representations can contain a certain amount of incorrect or contradictory data while still showing a good level of performance. This turns out to be an asset when a large lexicon is to be constructed automatically. A number ....

Hinton, G. E., McClelland, J. L. and Rumelhart, D. E. (1986). Distributed Representations. In D. E. Rumelhart and J. L. McClelland (Eds) Parallel Distributed Processing: Explorations in the microstructure of cognition. Volume I: foundations (pp. 77-109). Cambridge MA: MIT Press.

....features. Electrophysiological recordings demonstrate that distributed representations are widely used in the cortex (e.g. 3, 4, 5] The functional benefits of distributed representations include greater efficiency, robustness, and accuracy, and the ability to represent similarity relationships [6]. The efficiency of distributed representations can be appreciated by analogy with letters. Just as different combinations of a small number of letters can represent a large number of words, so can different combinations of a small set of units represent a large amount of information. The ....

G. E. Hinton, J. L. McClelland, and D. E. Rumelhart. Distributed representations. In D. E. Rumelhart, J. L. McClelland, and PDP Research Group, editors, Parallel Distributed Processing. Volume 1: Foundations, chapter 3, pages 77--109. MIT Press, Cambridge, MA, 1986.

....has father Pietro and Pietro has brother Giovanni , we would like to be able to infer Alberto has uncle Giovanni . Our approach is to learn appropriate distributed representations of the entities in the data, and then exploit the generalization properties of the distributed representations [2] to make the inferences. In this paper we present a method, which we have called Linear Relational Embedding (LRE) which learns a distributed representation for the concepts by embedding them in a space where the relations between concepts are linear transformations of their distributed ....

Geoffrey E. Hinton, James L. McClelland, and David E. Rumelhart. Distributed representations. In David E. Rumelhart, James L. McClelland, and the PDP research Group, editors, Parallel Distributed Processing, volume 1, pages 77--109. The MIT Press, 1986.

No context found.

G. E. Hinton, J. L. McClelland, and D. E. Rumelhart. Distributed representations. In D. E. Rumelhart and J. L. McClelland, editors, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, volume 1: Foundations, pages 77--109. MIT Press, Cambridge, MA, 1986.

No context found.

Hinton, G.E., J.L. McClelland & D.E. Rumelhart, 1986. Distributed Representations. In D.E. Rumelhart, J.L. McClelland and the PDP Research Group, eds. Parallel Distributed Processing: Experiments in the Microstructure of Cognition 1: Foundations (MIT Press, Cambridge, MA).

No context found.

G. E. Hinton, J. L. McClelland, and D. E. Rumelhart, "Distributed representations, " in Parallel Distributed Processing: Explorations In The Microstructure of Cognition, Volume 1: Foundations, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, 1986.

No context found.

G. E. Hinton, `Distributed representations', Technical report, Department of Computer Science, Carnegie Mellon University, Pittsburgh, PA, (1984).

No context found.

G.E. Hinton, J.L. McCelland, and D.E. Rumelhart. Distributed Representations. In McCelland Rumelhart, editor, Parallel Distributed Processing, volume 1. MIT Press, 1986.

No context found.

Hinton, G., McClelland, J., and Rumelhart, D. Distributed representations. In Parallel Distributed Processing: explorations in the microstructure of cognition, J. McClelland and D. Rumelhart, Eds., vol. I Foundations. MIT Press, 1986.

No context found.

G. E. Hinton, J. L. McClelland, and D. E. Rumelhart. Distributed representations. In Parallel Distributed Processing, volume 1, chapter 3, pages 77--109. MIT Press, Cambridge, MA, 1986.

No context found.

Hinton, G. E., McClelland, J. L. & Rumelhart, D. E. (1986), Distributed representations. In: Parallel Distributed Processing (Rumelhart, D. E., McClelland, J. L., eds.), vol. 1, Cambridge, MA: MIT Press, pp. 77-109.

No context found.

Hinton GE, McClelland JL, Rumelhart DE (1986) Distributed representations. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing, Vol 1. MIT Press, Cambridge, Mass., pp77--109

No context found.

Hinton GE, McClelland JL, Rumelhart DE (1986) Distributed representations. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing, vol 1. MIT Press, Cambridge, Mass, pp 77--109

No context found.

Hinton G. E., McClelland J. L. and Rumelhart D. E.: Distributed Representations, in: D.E. Rumelhart, J. L. McClelland, et al., Parallel Distributed Processing: Explorations in The Microstructure of Cognition, Vol. 1: Foundations, Cambridge: M.I.T. Press, 1986, pp. 77-109.

No context found.

D.E. Rumelhart, G.E. Hinton, and R.J. Williams. Distributed representations. In Parallel Distibuted Processing: Explorations in the Microstructure of Cognition, vol.1: Foundations. MIT Press, Cambridge, Massachuttes, 1986.

No context found.

G.E. Hinton, J.L. McClelland, and D.E. Rumelhart. Distributed representation. In D.E. Rumelhart and J.L. McClelland, editors, Parallel Distributed Processing. MIT Press, Cambridge, Massachusetts, 1986.

No context found.

Hinton, G. E., McClelland, J. L. and Rumelhart, D. E. (1986). Distributed Representations. In D. E. Rumelhart and J. L. McClelland (Eds) Parallel Distributed Processing: Explorations in the microstructure of cognition. Volume I: foundations (pp. 77-109). Cambridge MA: MIT Press.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC