K. Fukushima, N. Wake, "Handwritten alphanumeric character recognition by the neocognitron", IEEE Transactions on Neural Networks, 2[3], pp. 355--365, 1991  Home/Search   Document Not in Database   Summary   Related Articles   Check

....not as good as those produced by a neural network. Therefore a neural network is often seen as a universal tool to solve these problems. For example, in the field of pattern classification artificial neural networks have been employed very successfully to optical character recognition problems. [FW91] and [SKD92] show that even the fairly complex task of hand written character recognition can be successfully tackled by an artificial neural network. Even though these networks were only trained with very small sets of sample characters in various styles of handwriting, their ability to associate ....

Kunihiko Fukushima and Nobuaki Wake. Handwritten alphanumeric character recognition by the neocognitron. IEEE Transactions on Neural Networks, 2(3):355--365, May 1991.

....are seldom detailed in the relevant literature. We also present results which suggest that the selectivity parameters in the neocognitton can be adjusted in a straightforward manner so as to improve the classification performance of the neo cognitron. I Introduction Fukushima s neocognitron [2,3,4] has received attention over the past decade as a partially shift invariant [1,8] distortion tolerant classifier. It is one of the most complex artificial neural network structures to simulate, and this is perhaps the main reason that its performance has not been scrutinized to the extent of ....

....To assess the performance of a classifier to within 5 of its true value, with 95 confidence, it is necessary to perform at least 385 trials [6] We used a test set of 400 digits s to examine how a variety of parameters affected recognition rates. 3. 2 The Effect of the Gaussian Kernel Fukushima [4] and Menon et al. 8] describe two slightly different approaches to calculating the weights for the and d Gaussian kernels. If we let , be a twodimensional vector offset from the centre of the kernel then the weight at that position can be expressed as = 6) 7) where 0 t,t 1. Fukushima ....

[Article contains additional citation context not shown here]

K. Fukushima and N. Wake, "Handwritten alphanu- meric character recognition by the neocognitron," IEEE Trans. on Neural Networks, 2, no. 5, pp. 355365, May 1991.

....that decides the presence and absence of different primitive components in the character. In the Image Mapped approach the identification and the extraction of features are implicit processes within the recognition process. Neural Network based OCRs mostly follow this principle [Burr, 88] Fukushima, 91] Perantonis, 92] Guyon, 91] Schwenk, 95] Kunihito, 92] Hubrig Schaumburg, 92] Sabourin, 92] Grother, 92] Lovell, 94] The first reported work on OCR of Telugu Character is by Rajasekharan et al. [Rajasekhar, 77] This work identifies 50 primitive features and proposes a two stage ....

K. Fukushima and N. Wake. Handwritten alphanumeric character recognition by the neocognitron, IEEE Transactions on Neural Networks, vol. 2, pp. 355--65, May 1991.

....networks have shown convincing results in pattern recognition applications. For the task of invariant pattern recognition where an object has to be recognized regardless of its position and orientation in the image plane structured multilayer feedforward neural networks gained considerable success [2, 3, 9, 17]. Their characteristic is that a priori knowledge about the task to be performed is already built into the architecture by use of nodes with shared weight vectors. In [8] a class of structured invariant neural networks (SINN) has been proposed for shift and rotation invariant pattern recognition ....

K. Fukushima and N. Wake. Handwritten alphanumeric character recognition by the Neocognitron. IEEE Transactions on Neural Networks, 2(3):355365, May 1991.

....the construction of the learning algorithm leads to definite convergence after one training cycle only. 1 Introduction Structured feedforward neural networks have shown promising results in speech processing applications [9] and the field of shift and rotation invariant pattern recognition [1, 5, 6], see Fig. 1. Also higher order neural networks belong to this class of networks [2, 8] Their Figure 1: The same object: original, and shifted and rotated characteristic is that the invariance property is directly built into their architecture by the use of nodes with shared weight vectors. ....

....the constraints on the weights are too strong to allow an effective weight change. Thus the learning algorithm either ends up in the learning of a wrong target or it does not converge at all. If hard limiter transfer functions are used in the network a modified delta rule can be applied [1] for the training. Then the additional problem arises of finding suitable error functions for the hidden layers so that their minimization eventually leads to a minimum of the global error function. Here a new learning algorithm is proposed for a highly structured invariant neural network. First ....

K. Fukushima, N. Wake, Handwritten Alphanumeric Character Recognition by the Neocognitron, IEEE Trans. on Neural Networks, Vol. 2, No. 3, May 1991

....compared to a sequential computation of the SINN. 1 Introduction For the task of invariant pattern recognition where an object has to be recognized regardless of its position and orientation in the image plane structured multilayer feedforward neural networks gained considerable success [2, 3, 10, 17]. Their characteristic is that a priori knowledge about the task to be performed is already built into the architecture by use of nodes with shared weight vectors. In [6] a class of structured invariant neural networks (SINN) has been proposed for shift and rotation invariant pattern recognition ....

K. Fukushima, N. Wake. Handwritten alphanumeric character recognition by the Neocognitron. IEEE Trans. on Neural Networks, 2(3):355--365, May 1991.

.... systems [Marko, 1974, Jakubowicz, 1989, Le Cun et al. 1990, Cios and Shin, 1995, Hsieh and Chen, 1993, Fukumi et al. 1997, Hummel and Biederman, 1992, Rolls, 1994] The Neocognitron [Fukushima, 1975, Fukushima, 1979, Fukushima and Miyake, 1982, Fukushima et al. 1983, Fukushima, 1988a, Fukushima and Wake, 1991] a well known artificial neural network system designed for numeral and character recognition, is based on Hubel and Wiesel s findings. The supervised model [Fukushima et al. 1983] required substantial user interaction during feature selection and training. Hence, it can not efficiently be ....

....substantial user interaction during feature selection and training. Hence, it can not efficiently be applied to large and complex pattern sets. While learning efficiency can be improved through self organisation [Fukushima and Miyake, 1982] the recognition performance is generally reduced [Fukushima and Wake, 1991]. Both the supervised and the unsupervised model of the Neocognitron suffer from the need for extensive ad hoc tuning to application requirements. The performance of the network critically depends on numerous training parameters as well as the specific network configuration. In particular, the ....

[Article contains additional citation context not shown here]

Fukushima, K. and Wake, N. (1991). Handwritten alphanumeric character recognition by the neocognitron. IEEE Transactions on Neural Networks, 2(3):355-- 65.

....the parameters fl l ; ffi l ; ffi l are usually predefined, but since they affect the network s performance, ideally they should be determined during our training. For the mask distance j j, there are several implementations [Lovell, 1994] Here we will use Fukushima s approach described in [Fukushima and Wake, 1991, Lovell, 1994] That is, j j is the Euclidean distance to the centre of the receptive field, and the C cell mask is normalised as KSl X =1 X 2D l c l ( 1: 3 Trainable Network Parameters and Weights In this paper, we assume the network architecture (i.e. the number of planes, the ....

....linear in the Neocognitron, while a quadratic increase is observed in the MLP network. For large image sizes, the difference could be several orders of magnitude. Results presented in the literature as well as our experiments show that skillful feature selection improves the recognition rate [Fukushima and Wake, 1991]. However, manual feature selection can not be applied to large and complex pattern sets efficiently. Unfortunately, Fukushima Miyake s self organised version of the Neocognitron [Fukushima and Miyake, 1982] typically has even poorer generalisation performance. Elsewhere [Sabisch et al. 1998] ....

Fukushima, K. and Wake, N. (1991). Handwritten alphanumeric character recognition by the neocognitron. IEEE Transactions on Neural Networks, 2(3):355--65.

....characters. This problem is very popular in the pattern recognition community due to the wide scope of real world applications. Many various approaches have been proposed here in the literature, using statistics [32] structural syntactic methodology [33] 3] sophisticated neural networks [30][21] 22] or ad hoc feature extraction procedures [10] to mention only a few (for review, see [23] The data source was the MNIST database of handwritten digits (http: www.research.att.com yann ocr mnist) 23] which consists of two subsets, training and testing, containing together 70,000 ....

Wake, N. Handwritten Alphanumeric Character Recognition by the Neocognitron. IEEE Trans. on Neural Networks, 2(3), 1991, pp. 355-365.

....very popular in the pattern recognition community due to the wide scope of real world applications. Many various approaches have been proposed here in the literature, using statistics [Wong Chan 1998] structural syntactic methodology [Zabawa 1994] Cai Liu 1999] sophisticated neural networks [Wake 1991] [LeCun et al. 1989] LeCun Bengio 1994] or ad hoc feature extraction procedures [Kato, Omachi, et al. 1999] to mention only a few (for review, see [LeCun et al. 1995] Constructive induction in learning of image representation, K. Krawiec 7 Fig. 1. Exemplary images from the MNIST ....

Wake, N. Handwritten Alphanumeric Character Recognition by the Neocognitron. IEEE Trans. on Neural Networks, 2(3), 1991, pp. 355-365.

....Others extract topological features which depend on the global properties of the data. For example, Shridhar and Badreldin [7] use features derived from the character profiles in the image. They then feed these features into a tree classifier. More recently there have been a number [8] 9] [10] of successful attempts to automatically learn appropriate local features using feedforward neural networks. Some researchers [11] 4] 12] have boosted performance using combinations of classifiers. Significant progress has been made in OCR. On a standard database of lightly constrained ....

K. Fukushima and N. Wake, "Handwritten alphanumeric character recognition by the neocognitron ", IEEE Trans. Neural Networks, vol. 2, no. 3, pp. 355--365, 1991.

....words, there are not enough training points in the space created by the input images to allow accurate estimation of class probabilities throughout the input space. In the past, convolutional networks have been successively applied to zipcode recognition [11] and handwritten digit recognition [12]. A rotation invariant 11 Neocognitron has been successfully implemented and tested for the recognition of handwritten characters and digits [18] Convolutional networks (CN) provide an e cient method to constrain the complexity of feedforward neural networks by weight sharing and spatial ....

K. Fukushima and N. Wake, "Handwritten alphanumeric character recognition by the neocognitron," IEEE Transactions on Neural Networks, Volume 2, pages 355-65, 1991.

.... with any pool of binary features, which exhibit the appropriate invariance properties and the appropriate statistics on object and background, for example those suggested by Van Rullen et al. 1998) In the use of simple complex layers this architecture has similarities with Fukushima (1986) and Fukushima Wake (1991). Indeed in both papers the role of ORing in the complex cell layers as a means of obtaining invariance is emphasized. However there are major di erences between the architecture described here and the neocognitron paradigm. In our model training is only done for local features. The global ....

Fukushima, K. & Wake, N. (1991), `Handwritten alphanumeric character recognition by the neocognitron', IEEE Trans. Neural Networks.

....networks have shown convincing results in pattern recognition applications. For the task of invariant pattern recognition where an object has to be recognized regardless of its position and orientation in the image plane structured multilayer feedforward neural networks gained considerable success [2, 3, 9, 17]. Their characteristic is that a priori knowledge about the task to be performed is already built into the architecture by use of nodes with shared weight vectors. In [8] a class of structured invariant neural networks (SINN) has been proposed for shift and rotation invariant pattern recognition ....

K. Fukushima and N. Wake. Handwritten alphanumeric character recognition by the Neocognitron. IEEE Transactions on Neural Networks, 2(3):355365, May 1991.

....experts. It is quite tricky to choose a good structure to fit for the learning task at hand. This makes the advantages of connectionist learning much less attractive. Furthermore, it turns out that the structure of a neural network is closely related to its final performance in some applications [ Fukushima and Wake, 1991 ] Therefore, a method to build a neural network automatically from the training data should be developed. In this paper, a method to learn neural trees from training examples directly is proposed based on our early work [ Wen et al. 1992; Fang et al. 1991 ] The neural tree learned by this ....

K. Fukushima and N. Wake. Handwritten alphanumeric character recognition by the neocognition. IEEE Trans. on Neural Networks, 2, No. 3:355--365, 1991.

....uniform structure (such as a grid structure) of SONN cannot be a perfect panacea for all applications. For different applications, ideally, different networks structures should be carefully designed. SGNN SGNT gives a possible alternative to heavily handcrafted network methods, such as Neocognitron[3]. It automatically adjusts the neuron link density in different parts of the network according to the training sample distribution for a particular application, whereas SONN treats all applications uniformly and consequently, wastes neurons links in some parts of the network while has too few ....

....text is still fairly difficult. In all practical cases of network application to large problems, a strict network structure is constructed by the network designers. For example the AT T zip code network has seven layers and a complex hierarchical structure [6] More well known is the Neocognitron [3]. The success of the neocognitron derives very much from its structure and its careful training regime wherein the feature set is at a progressively higher level of abstraction towards the top of the network. So it appears that to tackle large problems some structuring of the problem is ....

K. Fukushima and N. Wake. Handwritten alphanumeric character recognition by the neocognition. IEEE Trans. on Neural Networks, 2, No. 3:355--365, 1991.

....the parameters l ; l ; l are usually prede ned, but since they a ect the network s performance, ideally they should be determined during our training. For the mask distance j j, there are several implementations [Lovell, 1994] Here we will use Fukushima s approach described in [Fukushima and Wake, 1991, Lovell, 1994] That is, j j is the Euclidean distance to the centre of the receptive eld, and the C cell mask is normalised as KSl X =1 X 2D l c l ( 1: 3 Trainable Network Parameters and Weights In this paper, we assume the network architecture (i.e. the number of ....

....linear in the Neocognitron, while a quadratic increase is observed in the MLP network. For large image sizes, the di erence could be several orders of magnitude. Results presented in the literature as well as our experiments show that skillful feature selection improves the recognition rate [Fukushima and Wake, 1991]. However, manual feature selection can not be applied to large and complex pattern sets eciently. Unfortunately, Fukushima Miyake s self organised version of the Neocognitron [Fukushima and Miyake, 1982] typically has even poorer generalisation performance. Elsewhere [Sabisch et al. 1998] we ....

Fukushima, K. and Wake, N. (1991). Handwritten alphanumeric character recognition by the neocognitron. IEEE Transactions on Neural Networks, 2(3):355-65.

....more refined neural network, to perform recognition directly with the images [24, 23] Networks can be cascaded to enhance the performance where the first level network has only to identify subfeatures, while higher layer networks assemble them to perform the CHAPTER 1. INTRODUCTION 5 recognition [35, 90, 94]. Alternatively, the networks can be fed feature vectors, such as a vector of Fourier descriptors, from which the network can be trained [59] Hidden Markov Models represent another kind of fully trainable algorithm. Simply stated, a HMM is a finite state automaton where the transitions between ....

K. Fukushima and N. Wake. Handwritten alphanumeric character recognition by the neocognitron. IEEE Transactions on Neural Networks, 2(3):355--365, May 1991.

....of the feature, thus enabling a simple computational scheme for detecting flexible spatial conjunctions. Furthermore the disjunction regions are not learned but hardwired into the architecture. In the use of simple complex layers this architecture has similarities with Fukushima (1986) and Fukushima Wake (1991). Indeed in both papers the role of ORing in the complex cell layers as a means of obtaining invariance is emphasized. However there are major differences between the architecture described here and the neocognitron paradigm. In our model training is only done for local features. The global ....

Fukushima, K. & Wake, N. (1991), `Handwritten alphanumeric character recognition by the neocognitron', IEEE Trans. Neural Networks.

....of practical import to neocognitron based systems. 2 Overview of the neocognitron The neocognitron is a complex model and it would be difficult to describe it sufficiently in the space allotted to this paper. For the sake of brevity, the reader is referred to Fukushima s thorough descriptions [6, 7] for details, both about the network and the system of notation which we shall adopt throughout this work. In general terms though, the neocognitron classifies input through a succession of functionally equivalent stages. Each stage extracts appropriate features from the output of the preceding ....

....selectivity parameters. However, SHOP does depend on a four layer architecture of the neocognitron (see e.g. 6] so that r 1 and r 4 can remain fixed whilst performance is optimized with respect to r 2 and r 3 . There are three points to note. ffl We assume that Fukushima s value of r 1 = 1:7 [7] is satisfactory. In practice, the function of first layer S cells is to extract fragments of lines from the input pattern. Observation indicates that r 1 1:7 results in spurious S cell responses and r 1 1:7 causes the S cells to become unresponsive. ffl The classification of the input is ....

K. Fukushima and N. Wake, "Handwritten alphanumeric character recognition by the neocognitron," IEEE Transactions on Neural Networks, vol. 2, no. 5, pp. 355--365, May 1991.

.... It does not apply to the general thresholded linear correlation classifier model, and its closed form training algorithm, which were also presented in [1] 2 Comparative Test Results We measured the classification performance of a NC that had been trained using the methods described by Fukushima [2, 3] and compared it to an identically structured NC trained using Hildebrandt s closed form training algorithm [1] In the following, these networks are referred to as NC F and NC H, respectively. In order to comply with Hildebrandt s definition of an optimal thresholded linear correlator, the ....

....of the S cells in the NC H network were chosen to maximize generalization whilst maintaining a discrimination index of unity 1 . The network structure and the training patterns that we used are described in [2] The values of the selectivity parameter used in the NC F network are given in [3], and the Gaussian kernels (used to weight the S cell and C cell inputs in both the NC F and NC H networks) were calculated using the methods described in that paper. 1 we use the terms generalization and discrimination as defined in [1] Network Correctly Classified Misclassified ....

Fukushima, K., & Wake, N. (1991). Handwritten Alphanumeric Character Recognition by the Neocognitron. IEEE Transactions on Neural Networks, 2(3), 355-365.

....has any advantages or disadvantages when compared with the activation functions studied in section 4.2. 5.2. 4 Character recognition in the neocognitron The neocognitron has been trained to recognize both handwritten Hangul by Kim and Lee [9] and alphanumeric characters by Fukushima and Wake [12]. In [12] the neocognitron was trained to recognize thirty ve 19 19 handwritten alphanumeric characters. The features learned at all levels of the network were handpicked, in order to optimize the training and recognition process. For Hangul recognition in [9] the neocognitron was trained to ....

....any advantages or disadvantages when compared with the activation functions studied in section 4.2. 5.2. 4 Character recognition in the neocognitron The neocognitron has been trained to recognize both handwritten Hangul by Kim and Lee [9] and alphanumeric characters by Fukushima and Wake [12] In [12], the neocognitron was trained to recognize thirty ve 19 19 handwritten alphanumeric characters. The features learned at all levels of the network were handpicked, in order to optimize the training and recognition process. For Hangul recognition in [9] the neocognitron was trained to recognize ....

K. Fukushima and N. Wake, \Handwritten alphanumeric character recognition by the neocognitron," IEEE Transactions on Neural Networks, vol. 2, pp. 355-365, May 1991.

No context found.

K. Fukushima, N. Wake, "Handwritten alphanumeric character recognition by the neocognitron", IEEE Transactions on Neural Networks, 2[3], pp. 355--365, 1991

No context found.

K. Fukushima and N. Wake 1991, "Handwritten alphanumeric character recognition by the neocognitron, " IEEE Trans. On Neural Networks 2(3), 355--365.

No context found.

K. Fukushima and N. Wake, "Handwritten alphanumeric character recognition by the neocognition", IEEE Trans. Neural Networks, 2, pp. 355--365, 1991.

First 50 documents

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC