Matching words and pictures

DMCA

Matching words and pictures (2003)

Cached

Download Links

Other Repositories/Bibliography

DBLP

by Kobus Barnard , Pinar Duygulu , David Forsyth , Nando De Freitas , David M. Blei , Michael I. Jordan

Venue:	JOURNAL OF MACHINE LEARNING RESEARCH
Citations:	665 - 40 self

BibTeX

@ARTICLE{Barnard03matchingwords,
    author = {Kobus Barnard and Pinar Duygulu and David Forsyth and Nando De Freitas and David M. Blei and Michael I. Jordan},
    title = {Matching words and pictures},
    journal = {JOURNAL OF MACHINE LEARNING RESEARCH},
    year = {2003},
    volume = {3},
    pages = {1107--1135}
}

OpenURL

Abstract

We present a new approach for modeling multi-modal data sets, focusing on the specific case of segmented images with associated text. Learning the joint distribution of image regions and words has many applications. We consider in detail predicting words associated with whole images (auto-annotation) and corresponding to particular image regions (region naming). Auto-annotation might help organize and access large collections of images. Region naming is a model of object recognition as a process of translating image regions to words, much as one might translate from one language to another. Learning the relationships between image regions and semantic correlates (words) is an interesting example of multi-modal data mining, particularly because it is typically hard to apply data mining techniques to collections of images. We develop a number of models for the joint distribution of image regions and words, including several which explicitly learn the correspondence between regions and words. We study multi-modal and correspondence extensions to Hofmann’s hierarchical clustering/aspect model, a translation model adapted from statistical machine translation (Brown et al.), and a multi-modal extension to mixture of latent Dirichlet allocation

Keyphrases

image region joint distribution semantic correlate many application particular image region region naming new approach aspect model associated text latent dirichlet allocation interesting example statistical machine translation multi-modal extension detail predicting word multi-modal data mining multi-modal data set specific case object recognition translation model correspondence extension whole image access large collection

DMCA