DMCA
A tutorial on energy-based learning (2006)
| Venue: | PREDICTING STRUCTURED DATA |
| Citations: | 57 - 6 self |
BibTeX
@INPROCEEDINGS{LeCun06atutorial,
author = {Yann LeCun and Sumit Chopra and Raia Hadsell and Fu Jie Huang and et al.},
title = {A tutorial on energy-based learning},
booktitle = {PREDICTING STRUCTURED DATA},
year = {2006},
publisher = {MIT Press}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
Energy-Based Models (EBMs) capture dependencies between variables by associating a scalar energy to each configuration of the variables. Inference consists in clamping the value of observed variables and finding configurations of the remaining variables that minimize the energy. Learning consists in finding an energy function in which observed configurations of the variables are given lower energies than unobserved ones. The EBM approach provides a common theoretical framework for many learning models, including traditional discriminative and generative approaches, as well as graph-transformer networks, conditional random fields, maximum margin Markov networks, and several manifold learning methods. Probabilistic models must be properly normalized, which sometimes requires evaluating intractable integrals over the space of all possible variable configurations. Since EBMs have no requirement for proper normalization, this problem is naturally circumvented. EBMs can be viewed as a form of non-probabilistic factor graphs, and they provide considerably more flexibility in the design of architectures and training criteria than probabilistic approaches.
Keyphrases
energy-based learning several manifold graph-transformer network capture dependency energy-based model observed variable maximum margin markov network probabilistic approach scalar energy generative approach many learning model energy function conditional random field traditional discriminative intractable integral common theoretical framework unobserved one ebm approach non-probabilistic factor graph probabilistic model possible variable configuration proper normalization
