E. Fiesler, A. Choudry, and H. J. Caulfield. A weight discretization paradigm for optical neural networks. In Proc. of the Int. Cong. on Opt. Sc. and Engg., pages 164--173, Bellingham, Washington, 1990. SPIE.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....in the size of the hidden layer in the network. 1. Integer Weight Nets The response of a feedforward multilayer neural net having continuous weights (CWN) can be approximated with one having discrete weights in one of two ways [1] by either allowing the number of discrete levels to grow [2, 4], or increasing the number of hidden neurons. This paper, while discussing both, will emphasise the latter of the two approaches, and will mainly be concerned with weights having small integer values. The results suggest that in many cases, finite weight resolution can be offset by an increased ....

E. Fiesler, A. Choudry, and H. J. Caulfield. A weight discretization paradigm for optical neural networks. In Proc. of the Int. Cong. on Opt. Sc. and Engg., pages 164--173, Bellingham, Washington, 1990. SPIE.

....percentage of the resulting networks failed to perform correctly when the weights were discretised. This was a rather drastic example of the truncation technique. Truncating the last bit or two of high resolution weights should give more reasonable results. The Continuous Discrete Learning Method [18] follows a more fruitful strategy. In this method, a trained CWN is discretised according to a predetermined stair case function, and then trained again using the standard BP procedure. The discretisation training cycle is repeated until the network converges to a satisfactory solution. The ....

E. Fiesler, A. Choudry, and H. J. Caulfield. A weight discretization paradigm for optical neural networks. In Proceedings of the International Congress on Optical Science and Engineering, pages 164--173, Bellingham, Washington, 1990. SPIE.

....in performance as compared with those in Ref. 6 is expected to be small. 3 Discrete Non Negative Multilayer Perceptrons: Theory and Experiments In this section, the subtraction compensation method is described and how it can be combined with an adaptation of the weight discretization method [16]. The performance of these techniques is evaluated in a series of experiments on training all positive discrete MLPs with five different LCLV response curves as non linear activation functions. IDIAP RR 97 02 5 3.1 On Subtraction Compensation and Weight Discretization 3.1.1 Subtraction ....

....in an efficient way. Important additional advantages of such a device are increased processing speed, a linear mapping of the weights, and compactness [21] To obtain successful network learning in the presence of discrete weights, an existing weight discretization method based on backpropagation [16] (described in detail in the Appendix) has been adapted for a OMLP with optical thresholding. It is easily implemented, and is suited for chip inthe loop learning. It integrates well with the subtraction compensation technique and can handle a precision of as low as a few bits. The basic idea of ....

[Article contains additional citation context not shown here]

E. Fiesler, A. Choudry, and H. J. Caulfield, "A Weight Discretization paradigm for Optical Neural Networks," in Proceedings of the International Congress on Optical Science and Engineering, vol. SPIE 1281, pp. 164--173, SPIE, Bellingham, Washington (1990).

.... therefore been included in our training algorithm whereby all positive weights are implemented by including a subtractive feedback to enable useful optical neural networks [7] The training of the optical neural network is performed by the weight discretization and update algorithm described in [4]. 3 Experiment and Results Input digit 0 1 2 3 4 5 6 7 8 9 Pattern Computed 1.0 0.65 0.79 0.89 0.72 0.53 0.82 0.77 0.75 0.56 Optical: Gray scale 1.0 0.83 0.94 0.86 0.81 0.56 0.74 0.72 0.72 0.48 Optical: Binary 1.0 0.91 0.32 0.55 0.76 0.64 0.79 0.82 0.83 0.45 Table 1: ONN Recall performance: ....

E. Fiesler, A. Choudry, and H. J. Caulfield, "A Weight Discretization Paradigm for Optical Neural Networks, " in Proceedings of the International Congress on Optical Science and Engineering, vol. SPIE 1281, pp. 164--173, SPIE, Bellingham, Washington (1990).

....networks. Pich e 95] 6 10 2 Statistical analysis of the effects of weight errors upon an ensemble of multilayer networks. Table 1: Weight discretization in multilayer neural networks: off chip learning. Reference Accuracy # of Benchmarks Remarks (in bits) Artificial Real world [Fiesler 88] Fiesler 90] 2 3 3 Forward pass with discrete weights, backward pass with continuous weights. Marchesi 93] 3 4 1 1 Power of two weights in the forward pass and an adaptive learning rate. Tang 93] 3 4 1 Power of two weights and adaptive gain of the activation function. Table 2: Weight discretization ....

.... that in this way the limited precision only plays a role in the forward propagation pass and that floating point calculations can be used in the backward pass (Table 2) One of the first, and perhaps most successful, weight discretization techniques is of the chip in the loop kind [Fiesler 88] Fiesler 90] It is suitable for feedforward neural networks, easy to implement, and very flexible in that it can handle a large range of discretizations up to the precision of a few bits only (Table 2) The basic idea is to start with a normal neural network with continuous valued weights. These weights are ....

E. Fiesler, A. Choudry, and H. J. Caulfield. A Weight Discretization paradigm for Optical Neural Networks. Proceedings of the International Congress on Optical Science and Engineering, vol. SPIE 1281, pp. 164--173, SPIE, Bellingham, Washington, 1990, ISBN: 0-8194-0328-8.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC