J.B. Pollack. On Connectionist Models of Natural Language Processing. PhD thesis, University of Illinois. Urbana, IL, 1987b.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....of neural networks have recently been shown to be equivalent to Turing machines. For the latter type of result, constructions using a potentially infinite network are presented in [20, 35] and constructions using a finite network, but real valued neurons of arbitrary precision are presented in [35, 78, 89]. The construction in [89] is rather interesting, as there the required precision grows only linearly in the space requirement of the simulated Turing machine, and only very simple saturated linear transfer functions are used. In the McCulloch Pitts Kleene model and its extensions, the input ....

Pollack, J. On Connectionist Models of Natural Language Processing. Ph. D. Thesis, Univ. Illinois, Urbana, 1987.

....be presented here takes the empiricist, or connectionist position . The theory has been developed for a hypothetical nervous system, or machine, called a perceptron (1958b, p.387) 3. See note 2. 4. Such proofs have been given for a number of modern connectionist architectures, for example by Pollack 1987 and Siegelmann and Sontag 1992. The latter establish the existence of a network capable of simulating a finite tape universal Turing machine in linear time. They are able to give an upper bound on the size of the network: at most 1058 units are required. 5. The letter is held in the NPL archive, ....

Pollack, J.B. 1987. On Connectionist Models of Natural Language Processing. Ph.D. Dissertation, University of Illinois, Urbana.

....the Von Neumann architecture (processor memory) It has been shown that neural nets are TM equivalent [HS87, FG90] once the issue of infinite memory tape is settled. The infinite tape may be represented as an external tape, as unbounded number of network units, or as infinite precision numbers [Pol87] In terms of nondiscrete computation, various researchers have shown that three layer feedforward nets can approximate any smooth function with arbitrary precision (for example, see [HSW89, HSW90, Cyb89] This result can be trivially extended to see how continuous recurrent nets can approximate ....

J. B. Pollack. On connectionist models of natural language processing. PhD thesis, Computer Science Dept., Univ. of Illinois, Urbana, 1987. MCCS-87-100.

....of neural networks have recently been shown to be equivalent to Turing machines. For the latter type of result, constructions using a potentially infinite network are presented in [15, 29] and constructions using a finite network, but real valued neurons of arbitrary precision are presented in [29, 60, 68]. The construction in [68] is rather interesting, as there the required precision grows only linearly in the space requirement of the simulated Turing machine. In these models the input to a net is given as a sequence of pulses. Another convention, closer to the current applications point of ....

Pollack, J. On Connectionist Models of Natural Language Processing. Ph. D. Thesis, Univ. Illinois, Urbana, 1987.

....power, compared to our model which has only a finite number of neurons. There has been previous work concerned with computability by finite networks, however, starting with the classical work of McCulloch and Pitts cited above on finite automata. Another related result was due to Pollack[20]. Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [20] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was high order , that is, the activations ....

....with computability by finite networks, however, starting with the classical work of McCulloch and Pitts cited above on finite automata. Another related result was due to Pollack[20] Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [20] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was high order , that is, the activations were combined using multiplications as opposed to just linear combinations (as in Equation 1) That is, the value of x i is ....

[Article contains additional citation context not shown here]

J.B. Pollack, On Connectionist Models of Natural Language Processing , Ph.D. Dissertation, Computer Science Dept, Univ. of Illinois, Urbana, 1987.

....machines are computationally more powerful than PDAs. Different methods have been proposed for proving that recurrent neural networks are computationally as powerful as Turing machines. The proof presented in [22] uses second order weights and an infinite number of neurons. The method proposed in [19] uses a finite number of neurons of two different types and higherorder weights. A construction of a first order recurrent neural networks with sigmoid neuron discriminant functions and a fixed number of neurons has been proposed in [21] Theorem 14 Let Psi : f0; 1g f0; 1g be any ....

J. Pollack, On Connectionist Models of Natural Language Processing. PhD thesis, Computer Science Department, University of Illinois, Urbana, IL, 1987.

....software engineering points of view. Unfortunately the question is rather ambiguous. It is easily demonstrated that a simple neural network model can emulate a finite state machine (It suffices to prescribe neural circuits for FLIP FLOPs and NAND gates. or even an (infinitetape) Turing machine [4], so there is a sense in which a neural network model has as much representational power as any other computational model which is likely to be on offer. Thus the issue involves largely subjective judgements of which representations are easier or more natural than which. A further ambiguity ....

J. B. Pollack. On connectionist models of natural language processing. PhD thesis, Urbana: Computer Science Dept., Univ. of Illinois. (Also Tech. Report. MCCS-87-100, Computing Research Lab., NMSU, Las Cruces, NM.), 1987.

....by reducing the halting problem to a reachability problem. Many of the undecidability results presented in this section are based on simulations of Turing machines using dynamical systems of a particular type. Similar Turing machine simulations are possible by other types of devices; see, e.g. [Pollack, 1987] for simulation by high order neural nets 13 , Bournez and Cosnard, 1996] for simulation by analog automata, Garzon, 1995] for simulation by cellular automata, Siegelmann and Sontag, 1994] and [Siegelmann, 1998] for simulation by saturated linear systems that can involve arbitrary real (not ....

Pollack, J. B. (1987). On connectionist models of natural language processing, Ph.D. Thesis, Dept of Computer Science, University of Illinois, Urbana.

....Machines. One important area in which symbolic methods have recently been used in neural networks is that of representation, i.e. theoretically what computational structures neural networks are provably equivalent or not equivalent to. The earliest recent work in the area seems to be Pollack [151]. Not surprisingly, all neural network architectures do not have the same computational power. Siegelmann [177, 176] proves that some recurrent network architectures are at least Turing equivalent while Frasconi [49] shows the computational limitations of a local RNN architecture. Giles [61] and ....

....not give explicit insight into the geometric properties of particular encodings. A related line of work [196, 136] focuses on designing the geometry of specific trajectories of metric space computers, including many connectionist networks. One essential idea (proposed in embryonic form in Pollack [151, 152], Siegelmann and Sonntag [175] Wiles and Elman [218] Rodriguez, Wiles, and Elman, to appear [160] is to use fractals to organize recursive computations in a bounded metric space. Cris Moore [136] provides the first substantial development of this idea, relating it to the traditional practice of ....

J. B. Pollack. On Connectionist Models of Natural Language Processing. PhD thesis, Computer Science Department, University of Illinois, Urbana, 1987. Available as TR MCCS-87-100, Computing Research Laboratory, New Mexico State University, Las Cruces, NM.

....a hierarchy of formal languages and corresponding symbolic automata. While the inherently symbolic systems of the Chomsky hierarchy have proven useful for describing certain aspects of human languages, it has more recently been suggested that dynamical systems may provide a more complete model (Pollack, 1987; Elman, 1991; for a collection of articles on the issue, see Port and van Gelder, 1995) Results suggest that such systems may correspond to a di erent, though related, set of language classes, as compared to their symbolic counterparts (Moore, 1998) A better understanding of these relationships ....

Pollack, J. B. (1987). On connectionist models of natural language processing.

....of human languages (Shieber, 1985) Moreover, this global view of the structure of the parameter space presents a contrast to the local view provided by gradient descent methods and may be useful in learning. To be sure, previous analyses have shown how analog devices can simulate Turing machines ((Pollack, 1987), Siegelmann and Sontag, 1991) and even recognizenonrecursively enumerable languages ( Siegelmann, 1996) Moore, 1996) thus, mere proofs of computational capability at the lower, context free, level are not revealing. However, these prior analyses have focused on complexity classification ....

Pollack, J. B. (1987). On connectionist models of natural language processing. Ph.D. Thesis, Department of Computer Science, University of Illinois.

....with computability by finite networks, however. The classical result of McCulloch and Pitts ( MP43] in 1943 (and Kleene [Kle56] showed how to implement logic gates by threshold networks, and therefore how to simulate finite automata by such nets. Another related result was due to Pollack[Pol87] Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [Pol87] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was high order , that is, the activations ....

....[Kle56] showed how to implement logic gates by threshold networks, and therefore how to simulate finite automata by such nets. Another related result was due to Pollack[Pol87] Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [Pol87] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was high order , that is, the activations were combined (as in Equation 1.1) using multiplications as opposed to just linear combinations (as in Equation 1.2) ....

[Article contains additional citation context not shown here]

J. B. Pollack. On Connectionist Models of Natural Language Processing. PhD thesis, Computer Science Dept, Univ. of Illinois, Urbana, 1987.

....rational weights and activities of unlimited precision. Pollack hypothesized that multiplicative (i.e. nonlinear) connections are a critical, and underappreciated, component for neurally inspired computing, in particular, for general purpose computing. One could argue that this paper disproves Pollack s hypothesis. Hartley Szu (1987) argued that TM s are equivalent both to potentially infinite neural networks with finite state neurons, and to finite networks of neurons with a countable infinity of states. More recently, Garzon and Franklin (1989, 1990; Franklin Garzon, 1990) have shown that countably infinite neural nets ....

Pollack, J. B. (1987). On connectionist models of natural language processing (Ph.D. dissertation). Urbana, IL: University of Illinois; also report MCCS-87-100, Las Cruces, NM: New Mexico State University, Computing Research Laboratory.

....devices In particular, and ignoring issues of training and practicality of implementation, one would like to know if every problem that can be solved by a digital computer is also solvable in principle using a net. This question has been asked before in the literature. Indeed, Jordan Pollack ([7]) showed that a certain recurrent net model which he called a neuring machine, for neural Turing is universal. In his model, all neurons synchronously update their states according to a quadratic combination of past activation values. In general, one calls high order nets those in which ....

....corresponds to q s 7 1 2 q s 1 2 , while pop (decrement) corresponds to q s 7 2q s Gamma 1. Reading a stack is straighforward: if q s encodes the stack value, then oe(2q s ) 0 if and only if the stack is empty, and oe(2q s ) 1 otherwise. A different encoding of a stack, as in [7], cannot be read in this simple manner. The critical point is to show that the whole design can be integrated (stack operations and state transitions gated by states of control unit and symbols at tops of stacks) without introducing high order connections, that is, products. This is achieved ....

Pollack, J.B., On Connectionist Models of Natural Language Processing, Ph.D. Dissertation, Computer Science Dept, Univ. of Illinois, Urbana, 1987. (Available as MCCS-87-100, Computing Research Laboratory, NMSU, Las Cruces, NM.)

....networks can represent variable length input [23, 40, 61] Most connectionist networks can be trained with a specific learning rule based on the network architecture and the units used. Connectionist networks have been shown to model successfully a whole variety of language learning tasks [60, 72, 45, 32]. In addition, the combination or integration of connectionist networks with statistical and symbolic representations is an important field for natural language processing [33, 22, 79, 76, 80] From the viewpoint of knowledge engineering, it might be efficient to encode well known rules rather ....

.... in general one could start with [25, 51] Architectural issues of connectionist and hybrid connectionist systems are discussed in [68, 22, 23, 40, 61, 3, 21, 52, 19, 56, 80] Some representative references for semantic and syntactic analysis with connectionist networks can be found in [38, 50, 60, 75, 70, 79]. For references on cognitively oriented connectionist natural language processing some references are [14, 78, 69, 42, 12] 3 Statistical Approaches 3.1 Introduction With the recent trend for learning in natural language processing, statistical methods have gained new popularity, and are being ....

J. B. Pollack. On connectionist models of natural language processing. Technical Report PhD thesis, Technical Report MCCS-87-100, New Mexico State University, Las Cruces, NM, 1987.

....conflicting data on linguistic behavior better than any previously proposed model of sentence processing. Yet COMPERE is not the first sentence processing model to be configured as a single process interacting with independent knowledge sources. The localist or punctate connectionist models of Pollack (1987; Waltz and Pollack, 1985) and Cottrell (1985; Cottrell and Small, 1983) resemble COMPERE at a gross architectural level, but these models did not offer the range of explanation of different behaviors that COMPERE does; for example, these models do not recover from errors, nor can they deal with ....

Pollack, J. B. 1987. On connectionist models of natural language processing, Technical Report, MCCS-87-100, Computing Research Laboratory, New Mexico State University.

.... demand the potential to store and act upon any one of an infinite collection of internal representations (e.g. 5, 6] Inspired by the pioneering work of McCulloch and Pitts [7] several simulation proofs of universal computation in recurrent networks have appeared over the last few years [8, 9, 10, 11]. Each of these projects clearly supports the claim of representational adequacy for recurrent networks by showing that recursive computation can arise by design. These projects, however, provide little insight in the detection of computation occurring in a system. In other words, they don t tell ....

J. B. Pollack, On Connectionist Models of Natural Language Processing, University of Illinois at Urbana-Champaign, Ph.D. thesis, 1987.

....connectionist and symbolic modules in hybrid processing architectures can be loosely coupled, tightly coupled or completely integrated (see figure 1) In a loosely coupled hybrid architecture processing has to be completed in one module before the next module can begin. For instance, the WP model [57, 44] for phenomenologically plausible parsing used a symbolic chart parser to construct syntactic localist networks which were also connected to semantic networks. Although the network itself may be viewed as a purely connectionist architecture, the overall architecture is a hybrid processing ....

J. B. Pollack. On connectionist models of natural language processing. Technical Report PhD thesis, Technical Report MCCS-87-100, New Mexico State University, Las Cruces, NM, 1987.

....papers by McCulloch and Pitts ( 12] 9] how to implement logic gates by threshold networks, and therefore how to simulate finite automata by such nets. For us, however, nets are essentially analog computational devices, in accordance with models currently used in neural net practice. In [14], Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [14] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was highorder , that is, the activations ....

....to simulate finite automata by such nets. For us, however, nets are essentially analog computational devices, in accordance with models currently used in neural net practice. In [14] Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [14] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was highorder , that is, the activations were combined using multiplications as opposed to just linear combinations. Pollack left as an open question that of ....

Pollack, J.B., On Connectionist Models of Natural Language Processing , Ph.D. Dissertation, Computer Science Dept, Univ. of Illinois, Urbana, 1987.

....of neural networks have recently been shown to be equivalent to Turing machines. For the latter type of result, constructions using a potentially infinite network are presented in [20, 35] and constructions using a finite network, but real valued neurons of arbitrary precision are presented in [35, 78, 89]. The construction in [89] is rather interesting, as there the required precision grows only linearly in the space requirement of the simulated Turing machine, and only very simple saturated linear transfer functions are used. In the McCulloch Pitts Kleene model and its extensions, the input ....

Pollack, J. On Connectionist Models of Natural Language Processing. Ph. D. Thesis, Univ. Illinois, Urbana, 1987.

....processing behavior regarding long distance dependencies and center embedding. The major focus in extending connectionist methods for natural language parsing has been to develop techniques to enable more dynamic behavior. In order to support their model of semantic disambiguation, Waltz Pollack (1985) had to use a traditional chart parser as a front end to their network creation process. This inspired Pollack (1985) to explore true connectionist techniques in the form of multiplicative connections and context adjusting processing nodes for achieving the network dynamism necessary for parsing. ....

....methods for natural language parsing has been to develop techniques to enable more dynamic behavior. In order to support their model of semantic disambiguation, Waltz Pollack (1985) had to use a traditional chart parser as a front end to their network creation process. This inspired Pollack (1985) to explore true connectionist techniques in the form of multiplicative connections and context adjusting processing nodes for achieving the network dynamism necessary for parsing. However, the contribution is in the exploration of low level connectionist techniques, and not in the proposal of a ....

[Article contains additional citation context not shown here]

Pollack, J. (1985). "On Connectionist Models of Natural Language Processing." Technical Report MCCS-87-100, Computing Research Laboratory, New Mexico State University, New Mexico.

....power, compared to our model which has only a finite number of neurons. There has been previous work concerned with computability by finite networks, however, starting with the classical work of McCulloch and Pitts cited above on finite automata. Another related result was due to Pollack[20]. Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [20] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was high order , that is, the activations ....

....with computability by finite networks, however, starting with the classical work of McCulloch and Pitts cited above on finite automata. Another related result was due to Pollack[20] Pollack argued that a certain recurrent net model, which he called a neuring machine, is universal. The model in [20] consisted of a finite number of neurons of two different kinds, having identity and threshold responses, respectively. The machine was high order , that is, the activations were combined using multiplications as opposed to just linear combinations (as in Equation 1) That is, the value of x i is ....

[Article contains additional citation context not shown here]

J.B. Pollack, On Connectionist Models of Natural Language Processing , Ph.D. Dissertation, Computer Science Dept, Univ. of Illinois, Urbana, 1987.

No context found.

J.B. Pollack. On Connectionist Models of Natural Language Processing. PhD thesis, University of Illinois. Urbana, IL, 1987b.

No context found.

Jordan B. Pollack. On connectionist models of natural language processing. Ph.D. Thesis, Department of Computer Science, University of Illinois, 1987.

No context found.

Pollack, J. B. (1987). On connectionist models of natural language processing (Ph.D. dissertation). Urbana, IL: University of Illinois; also report MCCS-87-100, Las Cruces, NM: New Mexico State University, Computing Research Laboratory.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC