Cross-Modal Similarity Learning via Pairs, Preferences, and Active Supervision
BibTeX
@MISC{Zhen_cross-modalsimilarity,
author = {Yi Zhen and Piyush Rai and Hongyuan Zha and Lawrence Carin},
title = {Cross-Modal Similarity Learning via Pairs, Preferences, and Active Supervision},
year = {}
}
Share
 |
 |
 |
 |
||
OpenURL
Abstract
We present a probabilistic framework for learning pair-wise similarities between objects belonging to differ-ent modalities, such as drugs and proteins, or text and images. Our framework is based on learning a binary code based representation for objects in each modal-ity, and has the following key properties: (i) it can leverage both pairwise as well as easy-to-obtain relative preference based cross-modal constraints, (ii) the prob-abilistic framework naturally allows querying for the most useful/informative constraints, facilitating an ac-tive learning setting (existing methods for cross-modal similarity learning do not have such a mechanism), and (iii) the binary code length is learned from the data. We demonstrate the effectiveness of the proposed approach on two problems that require computing pairwise sim-ilarities between cross-modal object pairs: cross-modal link prediction in bipartite graphs, and hashing based cross-modal similarity search.
Keyphrases
active supervision cross-modal similarity learning probabilistic framework cross-modal object pair differ-ent modality pairwise sim-ilarities ac-tive learning setting useful informative constraint pair-wise similarity easy-to-obtain relative preference binary code cross-modal similarity search cross-modal constraint bipartite graph prob-abilistic framework cross-modal link prediction binary code length cross-modal similarity following key property
