• Documents
  • Authors
  • Tables
  • Log in
  • Sign up
  • MetaCart
  • DMCA
  • Donate

CiteSeerX logo

Advanced Search Include Citations
Advanced Search Include Citations | Disambiguate

DMCA

The information bottleneck method (1999)

Cached

  • Download as a PDF

Download Links

  • [arxiv.org]
  • [arxiv.org]
  • [www.kon.cs.tu-berlin.de]
  • [www.cs.huji.ac.il]
  • [www.princeton.edu]
  • [www.cis.upenn.edu]
  • [www.cs.huji.ac.il]
  • [www.cnbc.cmu.edu]
  • [www.cs.huji.ac.il]
  • [www.cc.gatech.edu]
  • [www.cnbc.cmu.edu]
  • [www.cc.gatech.edu]
  • [www.cc.gatech.edu]
  • [www.ee.columbia.edu]
  • [www.cc.gatech.edu]
  • [www.cc.gatech.edu]
  • [www.wisdom.weizmann.ac.il]
  • [www.ee.columbia.edu]
  • [www.cnbc.cmu.edu]

  • Other Repositories/Bibliography

  • DBLP
  • Save to List
  • Add to Collection
  • Correct Errors
  • Monitor Changes
by Naftali Tishby , Fernando C. Pereira , William Bialek
Citations:536 - 35 self
  • Summary
  • Citations
  • Active Bibliography
  • Co-citation
  • Clustered Documents
  • Version History

BibTeX

@MISC{Tishby99theinformation,
    author = {Naftali Tishby and Fernando C. Pereira and William Bialek},
    title = {The information bottleneck method},
    year = {1999}
}

Share

Facebook Twitter Reddit Bibsonomy

OpenURL

 

Abstract

We define the relevant information in a signal x ∈ X as being the information that this signal provides about another signal y ∈ Y. Examples include the information that face images provide about the names of the people portrayed, or the information that speech sounds provide about the words spoken. Understanding the signal x requires more than just predicting y, it also requires specifying which features of X play a role in the prediction. We formalize this problem as that of finding a short code for X that preserves the maximum information about Y. That is, we squeeze the information that X provides about Y through a ‘bottleneck ’ formed by a limited set of codewords ˜X. This constrained optimization problem can be seen as a generalization of rate distortion theory in which the distortion measure d(x, ˜x) emerges from the joint statistics of X and Y. This approach yields an exact set of self consistent equations for the coding rules X → ˜ X and ˜ X → Y. Solutions to these equations can be found by a convergent re–estimation method that generalizes the Blahut–Arimoto algorithm. Our variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere.

Keyphrases

information bottleneck method    distortion measure    short code    rich framework    joint statistic    convergent re estimation method    blahut arimoto algorithm    relevant information    coding rule    variational principle    self consistent equation    limited set    face image    signal processing    maximum information    constrained optimization problem    rate distortion theory    exact set   

Powered by: Apache Solr
  • About CiteSeerX
  • Submit and Index Documents
  • Privacy Policy
  • Help
  • Data
  • Source
  • Contact Us

Developed at and hosted by The College of Information Sciences and Technology

© 2007-2019 The Pennsylvania State University