Results 1  10
of
71
Towards a mathematical theory of cortical microcircuits
 PLOS COMPUTATIONAL BIOLOGY
, 2009
"... The theoretical setting of hierarchical Bayesian inference is gaining acceptance as a framework for understanding cortical computation. In this paper, we describe how Bayesian belief propagation in a spatiotemporal hierarchical model, called Hierarchical Temporal Memory (HTM), can lead to a mathema ..."
Abstract

Cited by 68 (0 self)
 Add to MetaCart
(Show Context)
The theoretical setting of hierarchical Bayesian inference is gaining acceptance as a framework for understanding cortical computation. In this paper, we describe how Bayesian belief propagation in a spatiotemporal hierarchical model, called Hierarchical Temporal Memory (HTM), can lead to a mathematical model for cortical circuits. An HTM node is abstracted using a coincidence detector and a mixture of Markov chains. Bayesian belief propagation equations for such an HTM node define a set of functional constraints for a neuronal implementation. Anatomical data provide a contrasting set of organizational constraints. The combination of these two constraints suggests a theoretically derived interpretation for many anatomical and physiological features and predicts several others. We describe the pattern recognition capabilities of HTM networks and demonstrate the application of the derived circuits for modeling the subjective contour effect. We also discuss how the theory and the circuit can be extended to explain cortical features that are not explained by the current model and describe testable predictions that can be derived from the model.
What and where: A Bayesian inference theory of attention
, 2010
"... In the theoretical framework described in this thesis, attention is part of the inference process that solves the visual recognition problem of what is where. The theory proposes a computational role for attention and leads to a model that predicts some of its main properties at the level of psychop ..."
Abstract

Cited by 36 (6 self)
 Add to MetaCart
In the theoretical framework described in this thesis, attention is part of the inference process that solves the visual recognition problem of what is where. The theory proposes a computational role for attention and leads to a model that predicts some of its main properties at the level of psychophysics and physiology. In our approach, the main goal of the visual system is to infer the identity and the position of objects in visual scenes: spatial attention emerges as a strategy to reduce the uncertainty in shape information while featurebased attention reduces the uncertainty in spatial information. Featural and spatial attention represent two distinct modes of a computational process solving the problem of recognizing and localizing objects, especially in difficult recognition tasks such as in cluttered natural scenes. We describe a specific computational model and relate it to the known functional anatomy of attention. We show that several wellknown attentional phenomena – including bottomup popout effects, multiplicative modulation of neuronal tuning
How the brain might work: A hierarchical and temporal model for learning and recognition
 STANFORD UNIVERSITY
, 2008
"... ..."
(Show Context)
A computational model of the cerebral cortex
 In Proceedings of AAAI05, 938–943
, 2005
"... Our current understanding of the primate cerebral cortex (neocortex) and in particular the posterior, sensory association cortex has matured to a point where it is possible to develop a family of graphical models that capture the structure, scale and power of the neocortex for purposes of associativ ..."
Abstract

Cited by 30 (4 self)
 Add to MetaCart
Our current understanding of the primate cerebral cortex (neocortex) and in particular the posterior, sensory association cortex has matured to a point where it is possible to develop a family of graphical models that capture the structure, scale and power of the neocortex for purposes of associative recall, sequence prediction and pattern completion among other functions. Implementing such models using readily available computing clusters is now within the grasp of many labs and would provide scientists with the opportunity to experiment with both hardwired connection schemes and structurelearning algorithms inspired by animal learning and developmental studies. While neural circuits involving structures external to the neocortex such as the thalamic nuclei are less well understood, the availability of a computational model on which to test hypotheses would likely accelerate our understanding of these circuits. Furthermore, the existence of an agreedupon cortical substrate would not only facilitate our understanding of the brain but enable researchers to combine lessons learned from biology with stateoftheart graphicalmodel and machinelearning techniques to design hybrid systems that combine the best of biological and traditional computing approaches.
Integrated information increases with fitness in the evolution of animats. PLoS
 Computational Biology
, 2011
"... One of the hallmarks of biological organisms is their ability to integrate disparate information sources to optimize their behavior in complex environments. How this capability can be quantified and related to the functional complexity of an organism remains a challenging problem, in particular sinc ..."
Abstract

Cited by 10 (7 self)
 Add to MetaCart
One of the hallmarks of biological organisms is their ability to integrate disparate information sources to optimize their behavior in complex environments. How this capability can be quantified and related to the functional complexity of an organism remains a challenging problem, in particular since organismal functional complexity is not welldefined. We present here several candidate measures that quantify information and integration, and study their dependence on fitness as an artificial agent (‘‘animat’’) evolves over thousands of generations to solve a navigation task in a simple, simulated environment. We compare the ability of these measures to predict high fitness with more conventional informationtheoretic processing measures. As the animat adapts by increasing its ‘‘fit’ ’ to the world, information integration and processing increase commensurately along the evolutionary line of descent. We suggest that the correlation of fitness with information integration and with processing measures implies that high fitness requires both information processing as well as integration, but that information integration may be a better measure when the task requires memory. A correlation of measures of information integration (but also information processing) and fitness strongly suggests that these measures reflect the functional complexity of the animat, and that such measures can be used to quantify functional complexity even
Scalable Inference in Hierarchical Generative Models
 In Proceedings of the Ninth International Symposium on Artificial Intelligence and Mathematics
, 2006
"... Borrowing insights from computational neuroscience, we present a family of inference algorithms for a class of generative statistical models specifically designed to run on commonlyavailable distributedcomputing hardware. ..."
Abstract

Cited by 9 (1 self)
 Add to MetaCart
(Show Context)
Borrowing insights from computational neuroscience, we present a family of inference algorithms for a class of generative statistical models specifically designed to run on commonlyavailable distributedcomputing hardware.
Learning invariant features using inertial priors
 ANNALS OF MATHEMATICS AND ARTIFICIAL INTELLIGENCE
, 2006
"... We address the technical challenges involved in combining key features from several theories of the visual cortex in a single coherent model. The resulting model is a hierarchical Bayesian network factored into modular component networks embedding variableorder Markov models. Each component network ..."
Abstract

Cited by 7 (2 self)
 Add to MetaCart
(Show Context)
We address the technical challenges involved in combining key features from several theories of the visual cortex in a single coherent model. The resulting model is a hierarchical Bayesian network factored into modular component networks embedding variableorder Markov models. Each component network has an associated receptive field corresponding to components residing in the level directly below it in the hierarchy. The variableorder Markov models account for features that are invariant to naturally occurring transformations in their inputs. These invariant features give rise to increasingly stable, persistent representations as we ascend the hierarchy. The receptive fields of proximate components on the same level overlap to restore selectivity that might otherwise be lost to invariance.
Prediction games in infinitely rich worlds
 In Utility Based Data Mining Workshop (UBDM at KDD
, 2006
"... categories, every experience would be new, and one couldn’t make sense of one’s world. Furthermore, higher intelligence requires large numbers of categories, perhaps millions and beyond. Acquiring and robust detection of categories appears to be a complex task as categories interrelate in complex w ..."
Abstract

Cited by 6 (5 self)
 Add to MetaCart
(Show Context)
categories, every experience would be new, and one couldn’t make sense of one’s world. Furthermore, higher intelligence requires large numbers of categories, perhaps millions and beyond. Acquiring and robust detection of categories appears to be a complex task as categories interrelate in complex ways and occur in diverse conditions. We may then ask: how can a system learn so many complex interrelated categories? We propose and explore an avenue that we call prediction games in infinitely rich worlds. In these games, the world is a source of an unlimited stream of information. The games are played by a prediction system that in effect repeatedly experiments with its world and learns from its experiments. The system converts its input stream from the world into a sequence of learning episodes for itself. Each learning episode consists of the system hiding parts of the input, guessing (predicting) them using the remainder of the input (the local context), and updating itself based on comparing its observations with its predictions. The goal of the system is to improve its