Results 1 - 10
of
43
Street-to-Shop: Cross-Scenario Clothing Retrieval via Parts Alignment and Auxiliary Set
, 2012
"... In this paper, we address a practical problem of cross-scenario clothing retrieval- given a daily human photo cap-tured in general environment, e.g., on street, finding simi-lar clothing in online shops, where the photos are captured more professionally and with clean background. There are large dis ..."
Abstract
-
Cited by 21 (6 self)
- Add to MetaCart
In this paper, we address a practical problem of cross-scenario clothing retrieval- given a daily human photo cap-tured in general environment, e.g., on street, finding simi-lar clothing in online shops, where the photos are captured more professionally and with clean background. There are large discrepancies between daily photo scenario and on-line shopping scenario. We first propose to alleviate the human pose discrepancy by locating 30 human parts detected by a well trained hu-man detector. Then, founded on part features, we propose a two-step calculation to obtain more reliable one-to-many similarities between the query daily photo and online shop-ping photos: 1) the within-scenario one-to-many similari-ties between a query daily photo and the auxiliary set are derived by direct sparse reconstruction; and 2) by a cross-scenario many-to-many similarity transfer matrix inferred offline from an extra auxiliary set and the online shopping set, the reliable cross-scenario one-to-many similarities be-tween the query daily photo and all online shopping photos are obtained. We collect a large online shopping dataset and a daily photo dataset, both of which are thoroughly labeled with 15 clothing attributes via Mechanic Turk. The extensive exper-imental evaluations on the collected datasets well demon-strate the effectiveness of the proposed framework for cross-scenario clothing retrieval.
Learning to Place New Objects in a Scene
"... Abstract—Placing is a necessary skill for a personal robot to have in order to perform tasks such as arranging objects in a disorganized room. The object placements should not only be stable but also be in their semantically preferred placing areas and orientations. This is challenging because an en ..."
Abstract
-
Cited by 20 (10 self)
- Add to MetaCart
Abstract—Placing is a necessary skill for a personal robot to have in order to perform tasks such as arranging objects in a disorganized room. The object placements should not only be stable but also be in their semantically preferred placing areas and orientations. This is challenging because an environment can have a large variety of objects and placing areas that may not have been seen by the robot before. In this paper, we propose a learning approach for placing multiple objects in different placing areas in a scene. Given point-clouds of the objects and the scene, we design appropriate features and use a graphical model to encode various properties, such as the stacking of objects, stability, object-area relationship and common placing constraints. The inference in our model is an integer linear program, which we solve efficiently via an LP relaxation. We extensively evaluate our approach on 98 objects from 16 categories being placed into 40 areas. Our robotic experiments show a success rate of 98 % in placing known objects and 82 % in placing new objects stably. We use our method on our robots for performing tasks such as loading several dish-racks, a bookshelf and a fridge with multiple items. 1 I.
IntentSearch:Capturing User Intention for One-Click Internet Image Search
, 2012
"... Abstract—Web-scale image search engines (e.g. Google Image Search, Bing Image Search) mostly rely on surrounding text features. It is difficult for them to interpret users ’ search intention only by query keywords and this leads to ambiguous and noisy search results which are far from satisfactory. ..."
Abstract
-
Cited by 18 (7 self)
- Add to MetaCart
(Show Context)
Abstract—Web-scale image search engines (e.g. Google Image Search, Bing Image Search) mostly rely on surrounding text features. It is difficult for them to interpret users ’ search intention only by query keywords and this leads to ambiguous and noisy search results which are far from satisfactory. It is important to use visual information in order to solve the ambiguity in text-based image retrieval. In this paper, we propose a novel Internet image search approach. It only requires the user to click on one query image with the minimum effort and images from a pool retrieved by text-based search are re-ranked based on both visual and textual content. Our key contribution is to capture the users ’ search intention from this one-click query image in four steps. (1) The query image is categorized into one of the predefined adaptive weight categories, which reflect users ’ search intention at a coarse level. Inside each category, a specific weight schema is used to combine visual features adaptive to this kind of images to better re-rank the text-based search result. (2) Based on the visual content of the query image selected by the user and through image clustering, query keywords are expanded to capture user intention. (3) Expanded keywords are used to enlarge the image pool to contain more relevant images. (4) Expanded keywords are also used to expand the query image to multiple positive visual examples from which new query specific visual and textual similarity metrics are learned to further improve content-based image re-ranking. All these steps are automatic without extra effort from the user. This is critically important for any commercial web-based image search engine, where the user interface has to be extremely simple. Besides this key contribution, a set of visual features which are both effective and efficient in Internet image search are designed. Experimental evaluation shows that our approach significantly improves the precision of top ranked images and also the user experience.
Learning Hierarchical Similarity Metrics
"... Categories in multi-class data are often part of an underlying semantic taxonomy. Recent work in object classification has found interesting ways to use this taxonomy structure to develop better recognition algorithms. Here we propose a novel framework to learn similarity metrics using the class tax ..."
Abstract
-
Cited by 17 (0 self)
- Add to MetaCart
(Show Context)
Categories in multi-class data are often part of an underlying semantic taxonomy. Recent work in object classification has found interesting ways to use this taxonomy structure to develop better recognition algorithms. Here we propose a novel framework to learn similarity metrics using the class taxonomy. We show that a nearest neighbor classifier using the learned metrics gets improved performance over the best discriminative methods. Moreover, by incorporating the taxonomy, our learned metrics can also help in some taxonomy specific applications. We show that the metrics can help determine the correct placement of a new category that was not part of the original taxonomy, and can provide effective classification amongst categories local to specific subtrees of the taxonomy. 1.
Semantic-aware Co-indexing for Image Retrieval
"... Inverted indexes in image retrieval not only allow fast access to database images but also summarize all knowledge about the database, so that their discriminative capacity largely determines the retrieval performance. In this paper, for vocabulary tree based image retrieval, we propose a semantic-a ..."
Abstract
-
Cited by 4 (0 self)
- Add to MetaCart
(Show Context)
Inverted indexes in image retrieval not only allow fast access to database images but also summarize all knowledge about the database, so that their discriminative capacity largely determines the retrieval performance. In this paper, for vocabulary tree based image retrieval, we propose a semantic-aware co-indexing algorithm to jointly embed two strong cues into the inverted indexes: 1) local invariant features that are robust to delineate low-level image contents, and 2) semantic attributes from large-scale object recognition that may reveal image semantic meanings. For an initial set of inverted indexes of local features, we utilize 1000 semantic attributes to filter out isolated images and insert semantically similar images to the initial set. Encoding these two distinct cues together effectively enhances the discriminative capability of inverted indexes. Such co-indexing operations are totally off-line and introduce small computation overhead to online query cause only local features but no semantic attributes are used for query. Experiments and comparisons with recent retrieval methods on 3 datasets, i.e., UKbench, Holidays, Oxford5K, and 1.3 million images from Flickr as distractors, manifest the competitive performance of our method 1. 1.
Multi-Class Open Set Recognition Using Probability of Inclusion
"... Abstract. The perceived success of recent visual recognition approaches has largely been derived from their performance on classification tasks, where all possible classes are known at training time. But what about open set problems, where unknown classes appear at test time? Intuitively, if we coul ..."
Abstract
-
Cited by 4 (3 self)
- Add to MetaCart
(Show Context)
Abstract. The perceived success of recent visual recognition approaches has largely been derived from their performance on classification tasks, where all possible classes are known at training time. But what about open set problems, where unknown classes appear at test time? Intuitively, if we could accurately model just the positive data for any known class without overfitting, we could reject the large set of unknown classes even under an assumption of incomplete class knowledge. In this paper, we formulate the problem as one of modeling positive training data at the decision boundary, where we can invoke the statistical extreme value theory. A new algorithm called the PI-SVM is introduced for estimating the unnormalized posterior probability of class inclusion. 1
Classemes and Other Classifier-based Features for Efficient Object Categorization
- SUBMITTED TO IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE
"... This paper describes compact image descriptors enabling accurate object categorization with linear classification models, which offer the advantage of being efficient to both train and test. The shared property of our descriptors is the use of classifiers to produce the features of each image. Intui ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
This paper describes compact image descriptors enabling accurate object categorization with linear classification models, which offer the advantage of being efficient to both train and test. The shared property of our descriptors is the use of classifiers to produce the features of each image. Intuitively, these classifiers evaluate the presence of a set of basis classes inside the image. We first propose to train the basis classifiers as recognizers of a hand-selected set of object classes. We then demonstrate that better accuracy can be achieved by learning the basis classes as “abstract categories” collectively optimized as features for linear classification. Finally, we describe several strategies to aggregate the outputs of basis classifiers evaluated on multiple subwindows of the image in order to handle cases when the photo contains multiple objects and large amounts of clutter. We test our descriptors on challenging benchmarks of object categorization and detection, using a simple linear SVM as classifier. Our results are on par with those achieved by the best systems in these fields but are produced at orders of magnitude lower computational costs and using an image representation that is general and not specifically tuned for a predefined set of test classes.
Attribute-augmented semantic hierarchy: Towards bridging semantic gap and intention gap in image retrieval
, 2013
"... This paper presents a novel Attribute-augmented Semantic Hierarchy (A2SH) and demonstrates its effectiveness in bridging both the semantic and intention gaps in Content-based Image Retrieval (CBIR). A2SH organizes the semantic concepts into multiple semantic levels and augments each concept with a s ..."
Abstract
-
Cited by 2 (0 self)
- Add to MetaCart
This paper presents a novel Attribute-augmented Semantic Hierarchy (A2SH) and demonstrates its effectiveness in bridging both the semantic and intention gaps in Content-based Image Retrieval (CBIR). A2SH organizes the semantic concepts into multiple semantic levels and augments each concept with a set of related attributes. The attributes are used to describe the multiple facets of the concept and act as the intermediate bridge connecting the concept and low-level visual content. A hierarchical semantic similarity function is learnt to characterize the semantic similarities among images for retrieval. To better capture user search intent, a hybrid feedback mechanism is developed, which collects hybrid feedbacks on attributes and images. These feedbacks are then used to refine the search results based on A2SH. We use A2SH as basis to develop a unified content-based image retrieval system. We conduct extensive experiments on a large-scale data set of over one million Web images. Experimental results show that the proposed A2SH can characterize the semantic affinities among images accurately and can shape user search intent precisely and quickly, leading to more accurate search results as compared to state-of-the-art CBIR solutions.
Web Image Prediction Using Multivariate Point Processes
"... In this paper, we investigate a problem of predicting what images are likely to appear on the Web at a future time point, given a query word and a database of historical image streams that potentiates learning of uploading patterns of previous user images and associated metadata. We address such a W ..."
Abstract
-
Cited by 1 (0 self)
- Add to MetaCart
(Show Context)
In this paper, we investigate a problem of predicting what images are likely to appear on the Web at a future time point, given a query word and a database of historical image streams that potentiates learning of uploading patterns of previous user images and associated metadata. We address such a Web photo prediction problem at both a collective group level and an individual user level. We develop a predictive framework based on the multivariate point process, which employs a stochastic parametric model to solve the relations between image occurrence and the covariates that influence it, in a globally optimal, flexible, and scalable way. Using Flickr datasets of more than ten million images of 40 topics, our empirical results show that the proposed algorithm is more successful in predicting unseen Web images than other candidate methods, including reasoning on semantic meanings only, a state-of-art image retrieval method, and a generative topic model.
