G. E. Hinton. Representing part-whole hierarchies in connectionist networks. In Proceedings of CogSci.88, LEA, 1988.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....or semantic structure, and there is no relation (except the orthographic one) between the feature Mary object that occurs in the set John agent; loves; Mary object and the feature Mary subject that occurs in the set Mary subject; loves; John object . See, for example, the discussion in Hinton (1987) of role specific descriptors that represent the conjunction of an identity and a role [by the use of 14 Connectionism and Cognitive Architecture which] we can implement part whole hierarchies using set intersection as the composition rule. See also, Hinton, McClelland and Rumelhart (1986 ....

.... that any natural language that can express the proposition that John loves the girl can also express the proposition that the girl loves John) This consequence of the proposal that role relations be handled by role specific descriptors that represent the conjunction of an identity and a role (Hinton, 1987) offers a particularly clear example of how failure to postulate internal structure in representations leads to failuree to capture the systematicity of representational systems. 27 Connectionism and Cognitive Architecture of a conspecific. etc. Conversely, when there is no such story to ....

[Article contains additional citation context not shown here]

Hinton, G. (1987) . Representing part-whole Hierarchies in connectionist networks. Proceedings AAAI-87. Palo Alto: Kaufman.

....this second reason, the problem is not solved by simply replacing the local representation with a distributed one. guage syntax and semantics will crucially depend on advances in the connectionist treatment of hierarchical representations, be they localist, structured models [8] or PDP mechanisms [14]. 7 Acknowledgements I wish to thank Jerry Feldman and Subutai Ahmad for many fruitful discussions, as well as comments on earlier versions of this paper. ....

Geoffrey E. Hinton. Representing partwhole hierarchies in connectionist networks. In COGSCI [5], pages 48--54.

....output unit. For example, 3 layer back propagation networks have learned to convert text to phonemes (Sejnowski and Rosenberg, 1987) learned to form the past tense of regular and irregular verbs (Rumelhart and McClellend, 1986) and learned to discover regularities hidden within relationships (Hinton, 1988), to name just a few. 3.4 Connectionist Representations In order to examine connectionist representations, we need the foundation of some basic ideas and a set of common terms. One of the most basic concepts in connectionism is that of pattern. A pattern is an ordered set of activation values. ....

....learned by being trained on example analogies in spatial domains. Could the figureground associating method work on non spatial analogies The next set of experiments was designed to explore this question. 5. 3 Syntactic Analogies All of the following experiments are based on a domain created by Hinton (1988). Hinton s family tree model is a connectionist network that learns about relations between people in two isomorphic families, one Italian and the other English. Hinton s network was trained to answer questions, such as Who is Victoria s mother , and Who is Alphonso s grandfather After being ....

Hinton, G. (1988). Representing part-whole hierarchies in connectionist networks. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society, pages 48-54. Erlbaum, NJ.

....encodings of these component phrases can then be presented to the decoder to yield their phrasal components. To get the structure of the entire sentence, simply continue this process until all of the phrases give null values for their specifier and complements. This is similar to the approach that Hinton (1988) takes in descending through a similarly represented part whole hierarchy. It is the relationship between the X Bar template and the lexical entry for the words in the sentences that makes an X Bar approach to parsing attractive. The template is simple, and its variations are constrained by the ....

Hinton, G. (1988). Representing part-whole hierarchies in connectionist networks. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society, pages 48--54, Montreal, Canada.

....adding a new group of units in the network. This can be very inefficient if the relations do not use all the roles. Second, the system cannot represent multiple relations among the roles, which makes it inadequate for representing complex structures. 3.2. 2 Learning family relations In later work Hinton (1986, 1988) demonstrates how a distributed connectionist network can learn the relations of two isomorphic family trees in one English and 4 A structure is complex if its constituents are recursively nested. one Italian family. The family relations are presented similar to Hinton s earlier network, as ....

....pops off the b terminal and the stack reminder z a . Finaly, the last element in the stack, z a , is given as input, which pops off the terminal a and the empty stack v. Hence, the entire sequence is reconstructed. Although the RAAM network is capable of encoding complex tree like structures (Hinton 1988), it has nevertheless some limitations. First, there is no way to assure that the test of binary ness always will work, since a non terminal may also pass the test. As a result, the RAAM network will produce incorrect results. Second, learning in RAAM is generally slow as the non terminal ....

Hinton, G. E. (1988). Representing part-whole hierarchies in connectionist networks.

....and representing structure in general is difficult with neural networks. There are three key ideas that allow us to approach the problem of representing visual schemas. First, part whole relationships between an object and its parts can be represented by connections between localist units (Hinton, 1988). When a unit representing a part of an object is activated, its activity propagates to the unit representing the whole object. This process corresponds to bottom up input. Conversely, through feedback connections from the object unit to the part units, the object can activate its parts, ....

Hinton, G.E., 1988, Representing part-whole hierarchies in connectionist networks, in Proc.

....analysis and the Response Module produces responses expected by the environment. Figures (e) and (f) indicate the activities of finescaled and coarse scaled Relative Position Maps (RPMs) when attention is focused at the position marked with . similar to that of Hinton s part whole hierarchies [11]. VISOR consists of three main components: the Low Level Visual Module (simulated using procedural programs) the Schema Module, and the Response Module. The architecture and operation of the first two modules will be described below. The Response Module will be discussed with schema learning ....

Hinton, G. E. (1988). Representing part-whole hierarchies in connectionist networks. In Proceedings of the 10th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.

....illustrate its operation on blocks world scenes. 2 The VISOR Architecture The overall architecture of VISOR is based on the separation of the what and where pathways in low level vision (Van Essen and Anderson 1990; Fig. 1) Its representation of hierarchical knowledge is similar to that of Hinton (1988). VISOR consists of three main components: the Low Level Visual Module (simulated using procedural programs) the Schema Module, and the Response Module. The architecture and operation of the first two modules will be described below. The Response Module will be discussed with schema learning ....

Hinton, G. E. (1988). Representing part-whole hierarchies in connectionist networks. In Proceedings of the 10th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.

....hierarchical or recursive structure, e.g. music, natural language parsing, event perception. To illustrate, the spider that ate the hairy fly is a noun phrase containing the embedded noun phrase the hairy fly. Understanding such multilevel structures requires forming reduced descriptions (Hinton, 1988) in which a string of symbols or states ( the hairy fly ) is reduced to a single symbolic entity (a noun phrase) We present a neural net architecture that learns to encode the structure of symbol strings via such reduction transformations. The difficult problem of extracting multilevel structure ....

Hinton, G. E. (1988). Representing part--whole hierarchies in connectionist networks. Proceedings of the Eighth Annual Conference of the Cognitive Science Society.

....how to account for the open ended nature of language, and the commitment to discrete and well defined representations may make it difficult to capture the richness and high dimensionality required for language representations. Another major approach involves the use of distributed representations (Hinton, 1988; Hinton, McClelland, Rumelhart, 1986; van Gelder, in press) together with a learning algorithm, in order to infer the linguistic representations. Models which have used the localist approach have typically made an a priori commitment to linguistic representations (such as agent, patient, ....

....to understand than localist representations. There has been some tendency to feel that their murkiness is intractable and that distributed entails unanalyzable. Although, in fact, there exist various techniques for analyzing distributed representations (including cluster analysis, Elman, 1990; Hinton, 1988; Sejnowski Rosenberg, 1987; Servan Schreiber, Cleeremans, McClelland, in press; direct inspection, Pollack, 1988; principal component phase state analysis, Elman, 1989; and contri Elman Page 4 bution analysis,Sanger, 1989) the results of such studies have been limited. These analyses ....

[Article contains additional citation context not shown here]

Hinton, G.E. (1988). Representing part-whole hierarchies in connectionist networks.

.... representational nativism . There are several versions of this form of nativism, all based on the idea that we are born with some knowledge of representation types and forms. One version is that we are born with distributed representation windows which capture various part whole relationships. Hinton (1988), for instance, identifies three distributed approaches concerning how to map part whole hierarchies onto finite parallel hardware. First, there is a 1:1 mapping between an entity and a unit, and to use the connection as a pointer. This localist method was used in a model (McClelland and ....

....that John disliked Mary , where what follows that is itself a sentence. The idea that an instance can be represented in different ways according to the focus of attention is . viewing the hardware as a window that can be moved up and down (in discrete steps) over the part whole hierarchy (Hinton, 1988, p. 52) The important property of this method of representation is that the pattern of activity that represents the current whole is different from the pattern of activity that represents the same object when seen as a part of some other whole. It is difficult to envisage a satisfactory ....

[Article contains additional citation context not shown here]

Hinton, G. E. (1988) Representing part-whole hierarchies in connectionist networks, Proceedings of the Tenth Annual Conference of the Cognitive Science Society , Montreal, Quebec, Canada, pp.

....structure (by concatenation) and the representation of an element of the structure (by features) this type of system cannot represent embedded structures such as John saw the nurse riding an elephant. A solution to the feature buffer dichotomy problem was anticipated and sketched out by Hinton [19], and involved having a reduced description for NURSE RIDING ELEPHANT which would fit into the constituent buffers along with patterns for JOHN and SAW. However, it was not immediately obvious how to develop such reduced descriptions. Instead, avant garde connectionist representations were ....

....are stored, and reliable facsimiles are created with the use of domain knowledge. The systematic patterns developed by RAAM are a very new kind of representation, a recursive, distributed representation, which seems to instantiate Hinton s notion of the reduced description mentioned earlier [19]. They combine apparently immiscible aspects of well understood representations: They act both like feature vectors with their fixed width and simple measures of similarity, and like pointers, so that, with simple efficient procedures their contents can be fetched. Even further, they act like ....

G. Hinton, Representing Part-Whole hierarchies in connectionist networks, Proceedings of the Tenth Annual Conference of the Cognitive Science Society, Montreal, 1988, 48-54.

....The encodings of component phrases can be presented to the decoder to give their phrasal components. To get the structure of the entire sentence, simply continue this process until all of the phrases give null values for their specifier and complements. This is similar to the approach that Hinton (1988) takes in descending through a similarly represented part whole hierarchy. It is the relationship between the X Bar template and the lexical entry for the words in the sentences that makes an X Bar approach to parsing attractive. The template is simple, and its variations are constrained by the ....

Hinton, G. 1988. Representing part-whole hierarchies in connectionist networks. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society, Montreal, Canada. 48--54.

....as units in a network. The weight of the connection between two units represents how likely the two components are to be present in a schema, and the activity pattern of the network encodes a schema instantiation. The network does not encode hierarchical relationships among the schemas. Hinton [7] described three methods for representing hierarchical knowledge. The second method is similar to the one used in VISOR. The units in the network are organized into different levels. The higher the level, the more complex is the object that the unit represents. Lower level units representing ....

Geoffrey E. Hinton. Representing part-whole hierarchies in connectionist networks. In Proceedings of 10th Annual Conference of Cognitive Science Society, pages 48--54, 1988.

No context found.

G. E. Hinton. Representing part-whole hierarchies in connectionist networks. In Proceedings of CogSci.88, LEA, 1988.

No context found.

Hinton, G. (1988). Representing part-whole hierarchies in connectionist networks. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society, pages 48-54. Erlbaum, NJ.

No context found.

G. E. Hinton. Representing part-whole hierarchies in connectionist networks. In Proceedings of CogSci-88, LEA, 1988.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC