J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. Journal of Parallel and Distributed Computing, 6(2):291--330, 1989. 397  Home/Search   Document Details and Download   Summary   Related Articles   Check

....processors and DSPs, ii) design of special purpose VLSI chips, and (iii) design of analog and mixed (analog and digital) architectures. There are six types of parallelism available in an ANN [161] Out of them, node level and weight level parallelism are frequently exploited. Ghosh and Hwang [75] investigate architectural requirements for simulating ANNs using massively parallel multiprocessors. They propose a model for mapping neural networks onto message passing multicomputers. Liu [146] presents an efficient implementation of backpropogation algorithm on the CM 5 that avoids explicit ....

J. Ghosh and K. Hwang. Mapping neural networks onto message passing multicomputers. Journal of Parallel and Distributed Computing, 6:291--330, 1989.

....classification, speech recognition, machine vision, optimization, matching, image restoration, and so forth. Many algorithmic mapping techniques to implement ANNs on the available parallel architectures considering the inherent parallelism of ANNs have been reported (El Amawy Kulasinghe, 1997; Ghosh Hwang, 1989; Kumar, Shekhar, Amin, 1994b; Kung Hwang, 1989; Lin, Prasanna, Przytula, 1991; Malluhi, Bayoumi, Rao, 1995; Nordstrom Svensson, 1992; Singer, 1990; Svensson Nordstrom, 1990; Wah Chu, 1990) A number of algorithms mapped onto various architectures were surveyed in (Nordstrom ....

.... data (SIMD) arrays (Lin, Prasanna, Przytula, 1991; Singer, 1990) one dimensional SIMD arrays (Svensson Nordstrom, 1990) the cascaded systolic ring arrays (Kung Hwang, 1989) the hypercube architectures (Kumar, Shekhar, Amin, 1994b; Malluhi, Bayoumi, Rao, 1995) the multicomputers (Ghosh Hwang, 1989; Wah Chu, 1990) and the multiple bus systems (El Amawy Kulasinghe, 1997) The mapping algorithms proposed in (Lin, Prasanna, Przytula, 1991) are efficient for a network topology as long as the interconnections among neurons are sparse. However, this scheme needed a large number of ....

Ghosh, J. & Hwang, K. (1989). Mapping neural networks onto message--passing multicomputers, J. Parallel and Distributed Computing, 6, 291 -- 330.

....L 10I 2 J [3, 4] 2.3 Parallel Formulations of Backpropagation Here we survey existing schemes to parallelize BP for the fully connected multi layer networks. In these networks, the adjacent layers are completely connected. For schemes that primarily deal with randomly sparse networks, see [7, 26, 27, 28, 29]. Parallelization schemes for BP can be classified into three broad categories: network partitioning schemes, pattern partitioning schemes, and hybrid schemes. The network partitioning schemes take advantage of the parallelism in the computation of node activations and node errors by distributing ....

....utilization and speed up in our future work. We would also like to generalize our schemes by utilizing different lengths and breadths of processor grids for different layers of the network. It would also be interesting to compare the parallelization techniques for randomly sparse networks [7, 26, 27, 28, 29]. with our scheme for non uniform networks. 9 Acknowledgement We would like to thank Prof. Joydeep Ghosh (University of Texas at Austin) George Karypis and Ananth Grama for their useful comments during the drafting of this paper, and Dr. Robert Benner of Sandia labs for giving access to nCUBE2. ....

J. Ghosh and K. Hwang. Mapping neural networks onto message passing multicomputers. Jr. Parallel and Distributed Computing, April 1989.

....communication will occur. Since data are distributed in the network, the memory requirements are moderate. However, it is difficult to place the neurons in such a way as to produce efficient implementations, which require both an evenly distributed computational load and few data communications [1][2]. The second variation is based on the fact that the computations in a neural network are basically matrix products. Matrix products are also the only operations inside the network which absolutely require data communications between the processors. The advantage of this approach is the fact that ....

J. Ghosh, K. Hwang, "Mapping Neural Networks onto Message-Passing Multi-computers", Journal of Parallel and Distributed Computing, vol. 6, pp. 291-330, 1989.

....or cubes (tree dimensional) etc: In all cases, even if the subgroups overlap in all layers, clustering the neurons according to equivalence classes decreases the complexity of the network graph. 6 Note, that this approach is orthogonal to the clustering method of Gosh and Hwang [3], initiated for asynchronous neural networks. In their work, dense regions of the neural network graph are identified, clustered, and mapped onto one processor. A combination is possible, for sure. ....

Joydeep Ghosh and Kai Hwang. Mapping Neural Networks onto Message Passing Multicomputers. Journal of Parallel and Distributed Computing, 6:291--330, 1989.

....onto various parallel machine models. A number of researchers have contributed to the area of parallel implementations of ANNs by designing and analyzing algorithms to map specific ANN models on to specific parallel architectures. A discussion of the related literature follows below. Ghosh et al. [23] discuss the requirements to efficiently implement a generic neural network model on a multicomputer. The discussion includes mapping strategies, and an analysis of simulations of the mappings. In a similar vein, Chu and Wah [7, 101] describe optimal mapping of the learning process in multi layer ....

J. Ghosh and K. Hwang. Mapping Neural Networks onto Message-Passing Multicomputers. Journal of Parallel and Distributed Computing, 6:291--330, 1989.

....to simulate such nets using parallel supercomputers. However, many of today s supercomputers, like the CM 5, have vector processor nodes typically with a high penalty for sparse memory access patterns. Sparse nets do not seem well suited for simulation on such machines. Hence, many papers (e.g. [1, 2]) deal with the efficient mapping of arbitrary sparse networks on highly parallel computers. 3] maps multi layer backpropagation networks (including sparse connections between layers) onto networks of transputers, by partitioning each layer among the processors. However, little effort has been ....

....to avoid confusion in describing the model our terminology will reflect the structured network case. Our assumption of uniformly random connectivity is not quite accurate, since units will tend to be more densely connected to certain sets of other units and less densely connected to others. [1] presents a more detailed and accurate model of connectivity, but one that would be considerably harder to analyze. In our model, the network consists of U units distributed over P processors with u = U=P units assigned to each processor. Each unit s input connections are uniformly randomly ....

J. Ghosh and K. Hwang. Mapping neural networks onto message passing multicomputers. J. Parallel Distrib. Processing, 6:291--330, 1988.

....when large ANNs are used in applications [9] It is therefore judicious to explore parallel implementations of ANN models on general purpose parallel machines. A body of work has appeared in the literature that explores the research issues in the parallel implementation of some specific ANN models [4, 7, 10, 12, 13, 17]. In this paper, we will discuss a particular biologically inspired model of brain function called the Dynamic Link Architecture (DLA) 14] The DLA is a self organized neural network model that has theoretically been shown to possess translation invariant object recognition capabilities that are ....

Joydeep Ghosh and Kai Hwang. Mapping Neural Networks onto Message-Passing Multicomputers. Journal of Parallel and Distributed Computing, 6:291--330, 1989.

....a sparsely connected multi layer network (where the input activations are the outputs from the previous layer) Random connectivity might be quite an appropriate model for pruned backprop networks. Structured sparse networks might exhibit more clustering structure, as in the model developed by ( GH88] However, such a clustered model would be even harder to analyze. Chapter 3 Basic Sparse Model Our basic network consists of U units distributed over P processors, with u = U=P units per processor. The probability that a unit has a connection from any other unit is p c . Assuming a uniform ....

....some applications, but on structured networks we might find clusters of higher connectivity. Many of our arguments (for instance, the fact that an all to all broadcast is more efficient than selective communication section 3.5) may need to be rethought. Perhaps a model similar to that used by ( GH88] may be more appropriate. One important case that we are currently considering is the sparse activation case i.e. the case in which most of the activations are zero. We will show that a considerably different representation scheme yields much better performance in this case. Another case that is ....

Joydeep Ghosh and Kai Hwang. Mapping neural networks onto message passing multicomputers. Journal of Parallel and Distributed Processing, 6:291--330, 1988.

....required by todays neural network applications, a wide range of high performance target platforms are proposed and being used. Some neuro simulators use parallel processor systems (Tollenaere [21] Richards [17] Recce [16] Goddard [5] some are targeted on super computers (Ghosh and Wang [4]) others on dedicated neuro asics (Theeten [20] Han [6] or use heterogeneous architectures (Duranton [2] Application specific tools and devices For most experiments using neural networks, the information produced by the tools mentioned above is adequate for making general statements about ....

....or organizes his data, provided that it can be described following the data description described below. typedef struct f char name[50] name of the data = int id; identification of the data = int type; type of the data = int dimension; dimension of the data, 1; 2; 3; 4] int depth[4]; for each dimension, nr of elements = int length; the number of features per element = int flat; flat array or n dimensional = double time; current time value of the data = double previous update; previous update of the data = double delta time; next update of the data = int ....

J. Ghosh and K. Hwang. Mapping Neural Networks onto Message-Passing Multicomputers. Journal of Parallel and Distributed Computing, 6:291--330, 1989.

....applications. Typically, architectures with a fixed number of nodes and fixed number of layers are fabricated. Many special purpose implementations of neural networks have been described in the literature. A survey of parallel architectures for neural networks is given in [35] Ghosh and Hwang [12] investigate architectural requirements for simulating ANNs using massively parallel multiprocessors. They propose a model for mapping neural networks onto message passing multicomputers. Liu [19] presents an efficient implementation of backpropagation algorithm on the CM 5 that avoids explicit ....

J. Ghosh and K. Hwang. Mapping neural networks onto message passing multicomputers. Journal of Parallel and Distributed Computing, 6:291--330, 1989.

....Groups Several researchers have studied the properties of large neural networks by aggregating them into interacting groups or clusters. Typically a group exhibits more homogeneity among its constituent neurons, has higher internal connectivity, and or is used to represent a particular hypothesis [GH89]. Well known examples include the neuronal groups of Edelman [Ede87] and the neural clusters used for distortion invariant pattern matching by von der Malsburg [vdM88] In this section, we use the macroscopic state equation of a neuron population derived in the previous section to study a simple ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. Journal of Parallel and Distributed Computing, 6:291--330, April, 1989.

No context found.

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291-330, April, 1989.

No context found.

J. Ghosh and K. Hwang,"Mapping Neural Networks onto Message-Passing Multicomputers, "Journal of Parallel and Distributed Processing , Vol.6, pp.291-330, April 1989.

....ANN classifiers when they are powerful enough to form minimum error decision regions, if they are properly tuned, and when sufficient training data is available. Practical characteristics such as training time, classification time and memory requirements, however, can differ by orders of magnitude [9]. Also, the classifiers differ in their robustness against noise, effects of small training sets, and in their ability to handle high dimensional inputs [2] A good review of probabilistic, hyperplane, kernel and exemplar based classifiers that discusses the relative merit of various schemes ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291--330, April, 1989.

.... trees, K nearest neighbor, Gaussian mixtures, and CART can be found in [23, 25] It is seen that most of these networks show comparable performance over a wide variety of classification problems, while providing a range of trade offs in training time, coding complexity and memory requirements [9, 23]. Neural networks are not magical . They do require that the set of examples used for training should come from the same (possibly unknown) distribution as the set used for testing the networks, in order to provide valid generalization and good performance on classifying unknown signals [4, 16] ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291--330, April, 1989.

.... have studied the properties of large neural networks by aggregating them into interacting groups or clusters [Ama72, Ede87, Ama90, CG93] Typically a group exhibits more homogeneity among its constituent neurons, has higher internal connectivity, and or is used to represent a particular hypothesis [GH89]. Well known examples include the neuronal groups of Edelman [Ede87] and the neural clusters used for distortion invariant pattern matching by von der Malsburg [vdM88] In this section, we apply methods in statistics to build a macroscopic neural model. Macroscopically, the behavior of a cell ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. Journal of Parallel and Distributed Computing, 6:291-- 330, April, 1989.

....[22] This suggests that better solutions can be obtained by gradually increasing the gain till a step function is attained. Newer models that are better at avoiding local minima or have faster convergence rates have been proposed and compared [23, 21] and their architectural demands analyzed [7]. Recently, some researchers have studied applications of artificial neural networks to control problems in switching and communications [2, 12, 19] The continuous Hopfield neural net model (also known as Grossberg additive model [10] has been used to design a controller for a packet switched, ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291--330, April, 1989.

....of the interpretation results often fluctuate more often than the interpretation itself, but it is not critical that the latest value of a confidence factor needs to be used. This is similar to the simulation of several artificial neural network models where some asynchrony can be tolerated (Ghosh and Hwang, 1989), and allows more overlap between computation and communication. Our processor mapping strategy has some similarities to the PESA I architecture (Kuo and Moldovan, 1992) Here, each of the matching processors first receives a copy of the Rete graph for the particular matching level they work on. ....

Ghosh, J. and Hwang, K., 1989, "Mapping Neural Networks onto message-passing multicomputers," Journal of Parallel and Distributed Computing, vol. 6, pp. 291-330, April, 1989.

No context found.

Ghosh, J. and Hwang, K. (1989) Mapping neural networks onto message-passing multicomputers. Journal of Parallel and Distributed Computing, 6:291-330.

.... researchers have studied the properties of large neural networks by aggregating them into interacting groups or clusters [Ama72] Ama90] Typically a group exhibits more homogeneity among its constituent neurons, has higher internal connectivity, and or is used to represent a particular hypothesis [GH89]. Well known examples include the neuronal groups of Edelman [Ede87] and the neural clusters used for distortion invariant pattern matching by von der Malsburg [vdM88] In this papers, we focus on groups that are comparable in size to cell assemblies [GBA89] Macroscopically, the behavior of a ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291--330, April, 1989.

....characterization of short duration acoustic signals is not available yet. There are several neural networks that show comparable performance over a wide variety of classification problems, while providing a range of trade offs in training time, coding complexity and memory requirements [7, 8]. Some of these networks, including the multilayered perceptron when augmented with weight decay strategies [9] and the elliptical basis function network introduced in this paper, are quite insensitive to noise and to irrelevant inputs [10] Moreover, a firmer theoretical understanding of the ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291--330, April, 1989.

....ANN classifiers when they are powerful enough to form minimum error decision regions, if they are properly tuned, and when sufficient training data is available. Practical characteristics such as training time, classification time and memory requirements, however, can differ by orders of magnitude [GH89]. Also, the classifiers differ in their robustness against noise, effects of small training sets, and in their ability to handle high dimensional inputs [BG92] A good review of probabilistic, hyperplane, kernel and exemplar based classifiers that discusses the relative merit of various schemes ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291--330, April, 1989.

....Groups Several researchers have studied the properties of large neural networks by aggregating them into interacting groups or clusters. Typically a group exhibits more homogeneity among its constituent neurons, has higher internal connectivity, and or is used to represent a particular hypothesis [GBA89, GH89]. Well known examples include the neuronal groups of Edelman [Ede87] and the neural clusters used for distortion invariant pattern matching by von der Malsburg [vdM88] In this section, we use the macroscopic state equation of a neuron population derived in the previous section to study a simple ....

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. J. of Parallel and Distributed Computing, 6:291--330, April, 1989.

No context found.

J. Ghosh and K. Hwang. Mapping neural networks onto message-passing multicomputers. Journal of Parallel and Distributed Computing, 6(2):291--330, 1989. 397

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC