Results 1 
3 of
3
Generalization Properties of Modular Networks: Implementing the Parity Function
, 2001
"... The parity function is one of the most used Boolean function for testing learning algorithms because both of its simple definition and its great complexity. Being one of the hardest problems, many different architectures have been constructed to compute parity, essentially by adding neurons in the h ..."
Abstract

Cited by 13 (9 self)
 Add to MetaCart
The parity function is one of the most used Boolean function for testing learning algorithms because both of its simple definition and its great complexity. Being one of the hardest problems, many different architectures have been constructed to compute parity, essentially by adding neurons in the hidden layer in order to reduce the number of local minima where gradientdescent learning algorithms could get stuck. We construct a family of modular architectures that implement the parity function in which, every member of the family can be characterized by the fanin max of the network, i.e., the maximum number of connections that a neuron can receive. We analyze the generalization ability of the modular networks first by computing analytically the minimum number of examples needed for perfect generalization and second by numerical simulations. Both results show that the generalization ability of these networks is systematically improved by the degree of modularity of the network. We also analyze the influence of the selection of examples in the emergence of generalization ability, by comparing the learning curves obtained through a random selection of examples to those obtained through examples selected accordingly to a general algorithm we recently proposed.
Recursion in grammar . . .
"... Recursion in grammar and performance In the last 50 years of cognitive science, linguistic theory has proposed more and more articulated structures, while computer science has shown that simpler, flatter structures are more easily processed. If we are interested in adequate models of human linguisti ..."
Abstract
 Add to MetaCart
Recursion in grammar and performance In the last 50 years of cognitive science, linguistic theory has proposed more and more articulated structures, while computer science has shown that simpler, flatter structures are more easily processed. If we are interested in adequate models of human linguistic abilities, models that explain the very rapid and accurate human recognition and production of ordinary fluent speech, it seems we need to come to some appropriate understandingoftherelationshipbetweentheseapparently opposing pressures for more and less structure. Here we show how the apparent conflict disappears when it is considered more carefully. Even when we regard the linguists ’ project as a psychological one, there is no pressure for linguists to abandon their rather deep structures in order to account for our easy production and recognition of fluent speech. The deeper, more recursive structures reflect insights into similarities among linguistic constituents and operations, but a processor can compute exactly these structures without the extra effort that deeper analyses might seem to require. To show how this works, we
1306 Generalization Properties of Modular Networks: Implementing the Parity Function
"... Abstract—The parity function is one of the most used Boolean function for testing learning algorithms because both of its simple definition and its great complexity. Being one of the hardest problems, many different architectures have been constructed to compute parity, essentially by adding neurons ..."
Abstract
 Add to MetaCart
(Show Context)
Abstract—The parity function is one of the most used Boolean function for testing learning algorithms because both of its simple definition and its great complexity. Being one of the hardest problems, many different architectures have been constructed to compute parity, essentially by adding neurons in the hidden layer in order to reduce the number of local minima where gradientdescent learning algorithms could get stuck. We construct a family of modular architectures that implement the parity function in which, every member of the family can be characterized by the fanin max of the network, i.e., the maximum number of connections that a neuron can receive. We analyze the generalization ability of the modular networks first by computing analytically the minimum number of examples needed for perfect generalization and second by numerical simulations. Both results show that the generalization ability of these networks is systematically improved by the degree of modularity of the network. We also analyze the influence of the selection of examples in the emergence of generalization ability, by comparing the learning curves obtained through a random selection of examples to those obtained through examples selected accordingly to a general algorithm we recently proposed. Index Terms—Circuit complexity, generalization, learning from examples, modular neural networks, parity function. I.