We show how to use “complementary priors ” to eliminate the explaining away effects that make inference difficult in densely-connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive version of the wake-sleep algorithm. After fine-tuning, a network with three hidden layers forms a very good generative model of the joint distribution of handwritten digit images and their labels. This generative model gives better digit classification than the best discriminative learning algorithms. The low-dimensional manifolds on which the digits lie are modelled by long ravines in the free-energy landscape of the top-level associative memory and it is easy to explore these ravines by using the directed connections to display what the associative memory has in mind. 1

We present an algorithm that induces a class of models with thin junction trees---models that are characterized by an upper bound on the size of the maximal cliques of their triangulated graph. By ensuring that the junction tree is thin, inference in our models remains tractable throughout the learning process. This allows both an efficient implementation of an iterative scaling parameter estimation algorithm and also ensures that inference can be performed efficiently with the final model. We illustrate the approach with applications in handwritten digit recognition and DNA splice site detection.

Abstract not found

We accept this thesis as conforming

A fundamental problem with appearance-based recognition is how to encode the perceptual similarity between images as images need to be grouped based on their perceptual similarity. In this paper, we employ a spectral histogram model for generic appearance-based recognition. A perceptual component is defined as the spectral histogram of a training image, which encodes all the images perceptually similar to the input image. The similarity between two perceptual components is measured as # 2 distance between the corresponding spectral histograms, which has been shown to be perceptually meaningful. Building on this representation, we use the nearest neighbor classifier to classify an unseen input image, where each object class is represented by the perceptual components of the training images. A distinctive advantage of our representation is that it can be applied to many recognition problems, including texture classification, face recognition, and 3D object recognition. 1

This paper presents our work toward the integration of a multisensor microsystem with wireless communication, using system-on-chip (SoC) methodology. Four different forms of microelectronic sensors have been fabricated on two separate 5 5mm 2 silicon chips measuring pH, conductivity, dissolved oxygen concentration, and temperature. The sensors are integrated with a sensor fusion chip comprising analog circuitry for sensor operation and signal amplification prior to digital decoding and transmission. The microsystem prototype will be packaged in a miniature capsule, which measures 16 mm 55 mm including batteries and dissipates 6.3 mW for a minimal life cycle of 12 h. Index Terms---Laboratory-on-a-chip (LoC), microsystem, multisensor array, system-on-chip (SoC), wireless communication.

Abstract not found

Abstract not found

We have created a logic-based, first-order, and Turingcomplete set of software tools for stochastic modeling. Because the inference scheme for this language is based on a variant of Pearl's loopy belief propagation algorithm, we call it Loopy Logic. Traditional Bayesian belief networks have limited expressive power, basically constrained to that of atomic elements as in the propositional calculus. Our language contains variables that can capture general classes of situations, events, and relationships. A Turing-complete language is able to reason about potentially infinite classes and situations, with a Dynamic Bayesian Network. Since the inference algorithm for Loopy Logic is based on a variant of loopy belief propagation, the language includes an Expectation Maximization-type learning of parameters in the modeling domain. In this paper we briefly present the theoretical foundations for our loopy-logic language and then demonstrate several examples of stochastic modeling and diagnosis.

We address the technical challenges involved in combining key features from several theories of the visual cortex in a single coherent model. The resulting model is a hierarchical Bayesian network factored into modular component networks embedding variable-order Markov models. Each component network has an associated receptive field corresponding to components residing in the level directly below it in the hierarchy. The variable-order Markov models account for features that are invariant to naturally occurring transformations in their inputs. These invariant features give rise to increasingly stable, persistent representations as we ascend the hierarchy. The receptive fields of proximate components on the same level overlap to restore selectivity that might otherwise be lost to invariance.