Smolensky, P. 1990. Tensor product variable binding and the representation of symbolic structures in a connectionist system, Artificial Intelligence 46(1-2), 159--216.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....this to be found it would involve merely reimplementing classical symbolic architectures in a neural substrate. In response, several techniques for representing symbolic structures in NNs appeared, most notably the Recursive Auto Associative Memory (RAAM) 14] Tensor Product Representations [16] and Holographic Reduced Representations (HRRs) 12, 13] These techniques allowed vectors representing the constituents of a symbol structure to be combined into a single vector representing the whole structure, and for this vector to be decoded into the vectors representing the original ....

.... to train, and can lose constituent information for structures with deeply embedded constituents [3] HRRs require the use of very large vectors [12] e.g. a few thousand elements in size) and Tensor Product Representations grow in size explosively with the number of constituents embedded in a tree [16]. Whilst some promising techniques have been developed recently, e.g. 4, 1] o ering faster training, better generalisation or smaller representations, none have yet been used in large scale tasks. It is necessary not only to try applying these techniques to larger tasks but also to try and ....

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Arti cial Intelligence, 46(1-2):159-216, 1990.

....in the same vector, under certain assumptions of linear independence. Hence, BoltzCONS seems to satisfy the connectionist requirement of being distributed, as well as the criteria of systematicity and compositionality laid out earlier. 1.3. 2 Tensor Products Smolensky s Tensor Product model[101] is a general connectionist framework for modeling role ller relations. In this framework, the bindings between a set of roles and their llers are represented as the tensor (outer) product of a ller vector and a role vector. Unlike SHRUTI, however, the Tensor Product model is not necessarily ....

....only simultaneously represent a comparatively small number of elements without sacri cing accuracy, but they require a comparatively large set of units to represent even those items. Although HRR s do not su er from the potentially prohibitive growth that Smolensky observes in Tensor Products [101], the reduced nature of HRRs requires a clean up operation (dot product) in order to recognize the extracted vectors obtained by the correlation operators. From a more theoretical standpoint, BoltzCONS, with its direct encoding of traditional symbolic operations CAR, CDR, PUSH, and POP, seems to ....

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Articial Intelligence, 46:159216, 1990.

....rule is discovered the action part of the rule is executed by storing new triples in or removing old triples from the working memory. For more details see [Touretzky and Hinton, 1988] Dolan and Smolensky [1988] have given a formal account of the behavior of Dcps by using tensor product techniques [Smolensky, 1990]. The most interesting part of Dcps is the organization of the working memory and it is worth to mention its properties as they are not easily achieved by standard AI technologies. The working memory consists of 2000 units each representing 256 triples, i.e. the em receptive field of each unit ....

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46:159--216, 1990.

....dynamics which we defined in a mathematically precise way in Def. 2 can be found in [32] however, the focus of this so called recursive reduced descriptions lies on the properties of the dynamics and no concrete implementation is suggested. The tensor construction (T) proposed by Smolensky in [40] provides a concrete implementation for the encoding and decoding. These are fixed mappings f and g, respectively, based on the tensor product of two vectors. Smolensky decomposes the problem of encoding structures into three subproblems: decomposing the structure via roles, e.g. the label and ....

....different way for the various approaches. Assumed the encoding and decoding is fixed as in T and HRR, it is to be shown that precisely the proposed encoding and decoding are proper codings. Since this question is specific for the respective approach we refer to the respective specific literature [31,40]. Assumed the encoding and decoding are trainable as in LRAAM and FA, it is to be shown that networks can perform proper encoding with sufficient resources. We will deal with this question in Section 4. Assumed the encoding and decoding is fixed then every learning task is a standard learning task ....

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1-2):159-216, 1990.

....critisism, during the last decade a number of architectures and algorithms were designed demonstrating the capability of connectionist approaches to generate structured representations as well. The best known examples include recursive autoassociative memory [2] tensor product based approaches [3,4] and systems using synchrony of firing to perform binding [5,6] In this paper, we propose an alternative structure representing model which incorporates the concept of self organization. It is based on a hierarchy of modified self organizing maps (SOM) with leaky integrating units. The key idea ....

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artif. Intelligence, 46:159--216, 1990.

....by connectionists that human thought to some degree obeys the so called compositionality and systematicity principles. Thus, much research has focused on explicitly achieving these principles in connectionist hardware, and also supplying connectionist explanations to them (Smolensky, 1987; Smolensky, 1990; van Gelder, 1990; Pollack, 1990; Chalmers, 1990; Niklasson and Sharkey, 1992; Niklasson and van Gelder, 1994; Phillips, 1994; The success of these connectionist counter examples was questioned by Hadley (1994a) His concern was that the success in many of them could be explained by how the ....

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, (46):159--216.

.... here is that they avoid the binding problem that is incurred in producing distributed representations of multiple instances (see Page, 2000; Sougn, 1998 for further discussion) More complex representation schemes such as temporal binding (Shastri Ajjanagadde, 1993) or tensor product binding (Smolensky, 1990) have been proposed to allow the use of distributed representations that can represent multiple entities simultaneously without interference. Method A simple recurrent network (Elman, 1990) was used for these simulations. The network was trained with back propagation to map from sequences of ....

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159216.

....auto and hetero associative memory networks so that associations are encoded in activation values rather than weights. This requires neurons to be connected in a sparse and very precise and regular fashion. For example, matrixbased memories, such as Willshaw ## ## s (1969) associative nets and Smolensky s (1990) tensor product memories, can be formulated as a network of sigma pi neurons that are connected in patterns corresponding to the computation of an outer product. Sigma pi units compute a sum of products of inputs. Convolution based memories such as Willshaw, Buneman, and Longuet Higgins s (1969) ....

....associations, and possibly involve them in further recursive associations, is essential in an information ######## ######### ######## ####### 12 processing system that must deal with dynamic and complex knowledge structures, as does the human brain. Various authors, including Murdock (1982) Smolensky (1990), Pollack (1990) Plate (1994a) Plate (1995) Plate (2000) Halford ## ## (1994) Halford ## ## (1998) Gayler (1998) and Kanerva (2000) have proposed schemes for encoding structured compositional knowledge in vector representations. These knowledge structures can represent diverse information ....

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. ######### ############ ## (1-2), 159{ 216.

....architectures must be capable of modeling them if they are to claim a complete foundation for cognition. Halford, Wilson, Guo, Gayler, Wiles, and Stewart (1994) showed how relationships between constituents are made explicit (i.e. readily accessible) using tensor networks developed by Smolensky (1990). In a tensor network, primeness is made explicit by binding (taking the outer product of) vectors representing numbers and a vector representing the unary relation is prime. Prime numbers belong to the set: Prime(N) f2, 3, 5, g. They are represented 23 by the rank two tensor (matrix) T P ....

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159--216.

....or with their parity. Some researchers supported the connectionist to the top view that symbol structures and symbol processing should emerge from the work of a neural network (called a unified approach in chapters 2 and 4 of this volume and connectionist symbol processing in (Pollack, 1990, Smolensky, 1990, Touretzky, 1990, Smolensky et al. 1992, Smolensky, 1995) while others supported the synergistic hybrid approach bringing together connectionist and symbolic machines in a single system or model (Hendler, 1989b, 1991, Lange Dyer, 1989, Sun, 1992) Strange enough no one suggested to build up ....

Smolensky, P. (1990). Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems. AI, vol. 46 (1-2).

....below, but for the moment, observe that nodes labelled cats , see , mice are representations of the concepts involved, and nodes labelled # , # , and # represent the roles agent , action , and patient , respectively. The dark, diamond shaped nodes are binding nodes (cf. Cottrell, 1985; Smolensky, 1990; Stevenson, 1994) Binding nodes are used to bind nodes together into a unified representation. Thus, the fact that see is bound to # indicates that the see concept plays the action role in the entire proposition being represented. The node labelled core serves as a focal point for the ....

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159--216.

....very poorly on the the 1 s addition problem. In other words, the network, successfully trained on the first set of items, unlearned that first set when it was trained on the second one. 6 In earlier publications, Brousse and Smolensky, 1989b) Brousse and Smolensky, 1989a) Brousse and Smolensky, 1990) we used the term virtual memory , instead. The term virtual generalization , which obviously refers to learning tasks where there is an underlying mapping or structure to generalize on, and which in addition does not require that the connectionist network studied have properties of a memory ....

Paul Smolensky. Tensor product variable binding and the representations of symbolic structures in connectionist networks. Artificial intelligence, 46:159--216, 1990.

....of cognition without merely implementing a so called classical architecture. Since that time quite a number of connectionist models have been put forward, either by their designers or by others, as in some measure demonstrating that the challenge can be met (e.g. Pollack, 1988, 1990; Smolensky, 1990; Chalmers, 1990; Niklasson and Sharkey, 1992; Brousse, 1993) Unfortunately, it has generally been unclear whether these models actually do have this implication (see, for instance, the extensive philosophical debate in Smolensky, 1988; Fodor and McLaughlin, 1990; van Gelder, 1990, 1991; ....

Smolensky P. 1990: Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems, Artificial Intelligence, 46, 159 - 216.

....dynamical automata informally. Section 2 provides a corresponding formal treatment. A fractal is a set of points which is self similar at arbitrarily small scales. The classic example is the Cantor set. Consider the following infinite series of sets. The first set in the series is the interval [0, 1]. The next is the result of removing the middle third of this interval, namely, the set [0, 1 3] 2 3, 1] The next is the result of removing the middle thirds of each of the contiguous intervals in the previous set. This process is repeated indefinitely. The set which is the limit of this ....

....triangle. 1.0 0.5 0.0 0.5 1.0 X original set for various values of x 0 and all n 2 N . 1 Thus, the Cantor Set contains arbitrarily small copies of itself. It is worth noting that, under the definition just given, many other less exotic sets are also fractals. For example, the line segment [0, 1] is a fractal; the real number line is a fractal; the geometric series, f 1 r n : n 2 Ng is a fractal. The bounded fractals are more useful for forming realistic implementations, so I ll focus on them here. Fractals can also exist in multiple dimensions. The Sierpinski Triangle (Figure (1) is ....

[Article contains additional citation context not shown here]

Paul Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 1990. Special issue REFERENCES 44 on Connectionist symbol processing edited by G. E. Hinton.

....a (so called) classical cognitive architecture. Since then a number of connectionist models have been put forward, either by their authors or others, as in some measure either meeting the challenge, or suggesting that the challenge can be met in principle (for the models, see Pollack 1988, 1990; Smolensky 1990; Chalmers 1990; Niklasson Sharkey 1993; Brousse 1993; etc. Whether these models can or do meet the challenge has been the subject of much philosophical debate (Smolensky 1988; van Gelder 1990, 1991; Fodor McLaughlin 1990; Sharkey Jackson 1992; McLaughlin 1993a, 1993b; Clark 1993; Matthews ....

Smolensky P., (1990), Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems, Artificial Intelligence, 46, pp 159 - 216.

....CA network. An example of encoding of a labeled graph on which the theoretical results are applied is given as well. 1 Introduction The concept of distributed reduced representations was introduced by Hinton [4] in order to allow a neural network to represent compositional structure (see also [12, 15, 17]) Concrete examples of distributed reduced representations are given by the Recursive Auto Associative Memory (RAAM) by Pollack [11, 12] and by the Holographic Reduced Representations of Plate [10] In particular, the RAAM model is able to generate reduced representations of lists and ....

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46:159--216, 1990.

.... follows: 4) SSP develops a mathematical formalism showing precisely how a mental representation can be simultaneously a fully distributed pattern of numerical activities at one level of anaiysis and the functional near equivalent of a symbolic structure when analyzed at a higher level [32, 54,561. Integrating Connectionist and Symbolic Computatiou b. SSP shows in mathematica] detail, illustrated by computer simulations, how mental processing can be simultaneously a massively parallel process of spreading activation at one level of analysis and, at a higher level, a kind of ....

.... illustrated by computer simulations, how mental processing can be simultaneously a massively parallel process of spreading activation at one level of analysis and, at a higher level, a kind of parallel holistic manipulation of symbolic structures even those containing recursire embedding [8, 32, 56] c. SSP and related connectionist research demonstrate that the overall effects of spreading activation can often be analyzed at a higher level as a process of optimization, in which a representation is constructed that maximizes a connectionist measure of welJ formedncss we call Harmony [4, ....

[Article contains additional citation context not shown here]

Paul Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist networks. Artificial Intelligence, 46:159-216, 1990.

....it is argued that these two paradigms are mutually compatible. Integrating the basic assumptions of the paradigms results in formal theories of grammar that centrally incorporate a certain degree of connectionist computation. Two such grammar formalisms Harmonic Grammar (Legendre, Miyata and Smolensky, 1990ab) and Optimality Theory (Prince and Smolensky, 1991, 1993) are briefly introduced to illustrate grammar based approaches to connectionist language research. The strengths and weaknesses of grammar based research and more traditional model based research are argued to be complementary, ....

....1982; Rumelhart and Zipser, 1985; Mozer, 1991. And of course discreteness of representations is also a central property of a number of connectionist techniques for embedding symbolic structures as patterns of activity (e.g. Touretzky, 1986; Touretzky and Hinton, 1988; Dolan, 1989; Pollack, 1990; Smolensky 1990). The conclusion must be that the PDP Principle concerning representations, 1)a, is consistent with both crucial discreteness and crucial non discreteness of mental representations. In similar vein, consider the second PDP Principle, 1)b: Mental processes are massively parallel transformations ....

[Article contains additional citation context not shown here]

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist networks. Artificial Intelligence, 46, 159-216.

No context found.

Smolensky, P. 1990. Tensor product variable binding and the representation of symbolic structures in a connectionist system, Artificial Intelligence 46(1-2), 159--216.

No context found.

Smolensky,P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159-- 216.

No context found.

Smolensky Paul. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46:159--216, 1990.

No context found.

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46(1-2):159--216, 1990.

No context found.

Paul Smolensky.Tensor product variable binding and the representation of symbolic structures in connectionist systems. Arti#cial Intelligence, 46#1#2#:159# 216, 1990.

No context found.

P. Smolensky. Tensor product variable binding and the representation of symbolic structures in connectionist networks. Artifical Intelligence, 46:159--216, 1990.

No context found.

Smolensky, P. (1990). Tensor Product Variable Binding and the Representation of Symbolic Structures in Connectionist Systems. Artificial Intelligence, 46(1-2), 159-216.

First 50 documents  Next 50

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC