Abstract

This article initiates a rigorous theoretical analysis of the computational power of circuits that employ modules for computing winner-take-all. Computational models that involve competitive stages have so far been neglected in computational complexity theory, although they are widely used in computational brain models, artificial neural networks, and analog VLSI. Our theoretical analysis shows that winner-take-all is a surprisingly powerful computational module in comparison with threshold gates (= McCulloch-Pitts neurons) and sigmoidal gates. We prove an optimal quadratic lower bound for computing winner-take-all in any feedforward circuit consisting of threshold gates. In addition we show that arbitrary continuous functions can be approximated by circuits employing a single soft winner-take-all gate as their only nonlinear operation. Our

Citations

315	The Complexity of Boolean Functions - Wegener - 1987
274	Corticonics: neural circuits of the cerebral cortex - Abeles, ed - 1991
153	The Handbook of Brain Theory and Neural Networks - Arbib - 1995
101	Multilayer feedforward networks with a nonpolynomial activation function can approximate any function - Leshno, Lin, et al. - 1993
79	The Synaptic Organization of the Brain - Shepherd - 1979
72	Winner-take-all networks of O(n) complexity - Lazzaro, Ryckebusch, et al. - 1989
46	Models of Computation, Exploring the Power of Computing - Savage - 1998
38	Competition and cooperation in neural nets - Amari, Arbib - 1977
38	Discrete Neural Computation - A Theoretical Foundation - Siu, Roychowdhury, et al. - 1995
34	Pattern formation, contrast control, and oscillations in the short term memory of shunting on-center off-surround networks - Ellias, Grossberg - 1975
28	Currentmode subthreshold MOS circuits for analog neural systems - Andreou, Boahen, et al. - 1991
24	A cortical model of winner- take-all competition via lateral inhibition, Neural Networks 5 - Couluip, Granger, et al.
23	Analog VLSI circuits for attention-based, visual tracking - Horiuchi, Morris, et al. - 1997
21	Computational sensor for visual tracking with attention - Brajovic, Kanade - 1998
20	arrival times: A highly efficient coding scheme for neural networks. Parallel processing in neural systems and computers - Spike - 1990
16	A winner-take-all mechanism based on presynaptic inhibition feedback - Yuille, Grzywacz - 1989
13	Analog VLSI circuits for primitive sensory attention - DeWeerth, Morris - 1994
10	A high-precision VLSI winnertake-all circuit for self-organizing neural networks - Choi, Sheu - 1993
8	Dynamics of a winner-take-all neural network - Kincaid, Cohen, et al. - 1996
7	K-winners-take-all circuit with O(N) complexity - Urahama, Nagao - 1995
6	A 2D neuromorphic VLSI architecture for modeling selective attention - Indiveri
4	The design, fabrication, and test of a new VLSI hybrid analog-digital neural processing element - DeYong, Findley, et al. - 1992
4	Computational architectures for attention,” in The Attentive - Niebur, Koch - 1998
3	Pulse coded winner-take-all networks - Meador, Hylander - 1994
1	A winnertake. all mechanism based on presynaptic inhibition - Yuille, Grzywacz - 1989