Quantization and Pruning of Multilayer Perceptrons: Towards Compact Neural Networks (1997)
| Citations: | 2 - 1 self |
BibTeX
@MISC{Lundin97quantizationand,
author = {Tomas Lundin and Perry Moerland},
title = {Quantization and Pruning of Multilayer Perceptrons: Towards Compact Neural Networks},
year = {1997}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
A connectionist system or neural network is a massively parallel network of weighted interconnections, which connect one or more layers of non-linear processing elements (neurons). To fully profit from the inherent parallel processing of these networks, development of parallel hardware implementations is essential. However, these hardware implementations often differ in various ways from the ideal mathematical description of a neural network model. It is, for example, required to have quantized network parameters, in both electronic and optical implementations of neural networks. This can be because device operation is quantized or a coarse quantization of network parameters is beneficial for designing compact networks. Most of the standard algorithms for training neural networks are not suitable for quantized networks because they are based on gradient descent and require a high accuracy of the network parameters Several weight discretization techniques have been developed to reduce the required accuracy further without deterioration of network performance. One of the earliest of these techniques [Fiesler-88] is further investigated and improved in this report. Another way to obtain compact networks is by minimizing their topology for the problem at hand. However, it is impossible to know a priori the size of such minimal network topology. An unsuitable
Keyphrases
towards compact neural network neural network multilayer perceptrons compact network network parameter parallel hardware implementation neural network model minimal network topology high accuracy non-linear processing element standard algorithm ideal mathematical description inherent parallel processing device operation quantized network coarse quantization hardware implementation network performance connectionist system various way weighted interconnection gradient descent optical implementation required accuracy parallel network
