Results 1  10
of
17
Learning to Label Aerial Images from Noisy Data
"... When training a system to label images, the amount of labeled training data tends to be a limiting factor. We consider the task of learning to label aerial images from existing maps. These provide abundant labels, but the labels are often incomplete and sometimes poorly registered. We propose two ro ..."
Abstract

Cited by 8 (1 self)
 Add to MetaCart
(Show Context)
When training a system to label images, the amount of labeled training data tends to be a limiting factor. We consider the task of learning to label aerial images from existing maps. These provide abundant labels, but the labels are often incomplete and sometimes poorly registered. We propose two robust loss functions for dealing with these kinds of label noise and use the loss functions to train a deep neural network on two challenging aerial image datasets. The robust loss functions lead to big improvements in performance and our best system substantially outperforms the best published results on the task we consider. 1.
Compete to Compute
, 2013
"... Local competition among neighboring neurons is common in biological neural networks (NNs). We apply the concept to gradientbased, backproptrained artificial multilayer NNs. NNs with competing linear units tend to outperform those with noncompeting nonlinear units, and avoid catastrophic forgettin ..."
Abstract

Cited by 6 (2 self)
 Add to MetaCart
(Show Context)
Local competition among neighboring neurons is common in biological neural networks (NNs). We apply the concept to gradientbased, backproptrained artificial multilayer NNs. NNs with competing linear units tend to outperform those with noncompeting nonlinear units, and avoid catastrophic forgetting when training sets change over time.
ANALYZING DRUM PATTERNS USING CONDITIONAL DEEP BELIEF NETWORKS
"... We present a system for the highlevel analysis of beatsynchronous drum patterns to be used as part of a comprehensive rhythmic understanding system. We use a multilayer neural network, which is greedily pretrained layerbylayer using restriced Boltzmann machines (RBMs), in order to model the conte ..."
Abstract

Cited by 5 (0 self)
 Add to MetaCart
(Show Context)
We present a system for the highlevel analysis of beatsynchronous drum patterns to be used as part of a comprehensive rhythmic understanding system. We use a multilayer neural network, which is greedily pretrained layerbylayer using restriced Boltzmann machines (RBMs), in order to model the contextual timesequence information of a drum pattern. For the input layer of the network, we use a conditional RBM, which has been shown to be an effective generative model of multidimensional sequences. Subsequent layers of the neural network can be pretrained as conditional or standard RBMs in order to learn higherlevel rhythmic features. We show that this model can be finetuned in a discriminative manner to make accurate predictions about beatmeasure alignment. The model generalizes well to multiple rhythmic styles due to the distributed statespace of the multilayer neural network. In addition, the outputs of the discriminative network can serve as posterior probabilities over beatalignment labels. These posterior probabilities can be used for Viterbi decoding in a hidden Markov model in order to maintain temporal continuity of the predicted information. 1.
Scaling Up Natural Gradient by Sparsely Factorizing the Inverse Fisher Matrix
"... Secondorder optimization methods, such as natural gradient, are difficult to apply to highdimensional problems, because they require approximately solving large linear systems. We present FActorized Natural Gradient (FANG), an approximation to natural gradient descent where the Fisher matrix is ..."
Abstract

Cited by 3 (0 self)
 Add to MetaCart
(Show Context)
Secondorder optimization methods, such as natural gradient, are difficult to apply to highdimensional problems, because they require approximately solving large linear systems. We present FActorized Natural Gradient (FANG), an approximation to natural gradient descent where the Fisher matrix is approximated with a Gaussian graphical model whose precision matrix can be computed efficiently. We analyze the Fisher matrix for a small RBM and derive an extremely sparse graphical model which is a good match to the covariance of the sufficient statistics. Our experiments indicate that FANG allows RBMs to be trained more efficiently compared with stochastic gradient descent. Additionally, our analysis yields insight into the surprisingly good performance of the “centering trick ” for training RBMs. 1.
Deep Neural Nets as a Method for Quantitative Structure−Activity Relationships
"... ABSTRACT: Neural networks were widely used for quantitative structure−activity relationships (QSAR) in the 1990s. Because of various practical issues (e.g., slow on large problems, difficult to train, prone to overfitting, etc.), they were superseded by more robust methods like support vector machin ..."
Abstract

Cited by 2 (0 self)
 Add to MetaCart
ABSTRACT: Neural networks were widely used for quantitative structure−activity relationships (QSAR) in the 1990s. Because of various practical issues (e.g., slow on large problems, difficult to train, prone to overfitting, etc.), they were superseded by more robust methods like support vector machine (SVM) and random forest (RF), which arose in the early 2000s. The last 10 years has witnessed a revival of neural networks in the machine learning community thanks to new methods for preventing overfitting, more efficient training algorithms, and advancements in computer hardware. In particular, deep neural nets (DNNs), i.e. neural nets with more than one hidden layer, have found great successes in many applications, such as computer vision and natural language processing. Here we show that DNNs can routinely make better prospective predictions than RF on a set of large diverse QSAR data sets that are taken from Merck’s drug discovery effort. The number of adjustable parameters needed for DNNs is fairly large, but our results show that it is not necessary to optimize them for individual data sets, and a single set of recommended parameters can achieve better performance than RF for most of the data sets we studied. The usefulness of the
Assessment of sequential boltzmann machines on a lexical processing task
 in European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning
, 2012
"... Abstract. Recently, a promising probabilistic model based on Boltzmann Machines, i.e. the Recurrent Temporal RBM, has been proposed. It is able to learn physical dynamics (e.g. videos of bouncing balls), however up to now it was not clear whether this ability could apply to symbolic tasks. Here we ..."
Abstract

Cited by 2 (2 self)
 Add to MetaCart
(Show Context)
Abstract. Recently, a promising probabilistic model based on Boltzmann Machines, i.e. the Recurrent Temporal RBM, has been proposed. It is able to learn physical dynamics (e.g. videos of bouncing balls), however up to now it was not clear whether this ability could apply to symbolic tasks. Here we assess its capabilities on learning graphotactic rules from a set of English words. It emerged that the model is able to extract local transition rules between items of a sequence, but it does not seem to be suited to encode a whole word. 1
Intelligent System Design
, 2011
"... Sorting batteries with deep neural networks Using deep belief networks and convolutionary neural networks for image classification. ..."
Abstract
 Add to MetaCart
(Show Context)
Sorting batteries with deep neural networks Using deep belief networks and convolutionary neural networks for image classification.
BIOINFORMATICS doi:10.1093/bioinformatics/btu277 Deep
"... Vol. 30 ISMB 2014, pages i121–i129 ..."
(Show Context)
Deep learning approaches to problems in speech recognition, computational chemistry, and natural language text processing
, 2015
"... ..."