Existing content-based image retrieval paradigms almost never address the problem of starting the search, when the user has no starting example image but rather a mental image.

This paper introduces a tangible user interface for browsing and retrieving images from an image database. The basis for the query to the image database is a color layout sketch, which is used by the underlying query algorithm to find the best matches. The users are provided with colored cubes of various sizes (1.5 to 4 cm) and colors (8 base colors). The users can place and arrange the colored cubes on a small table to create a color layout sketch. Multiple users can use this interface to collaborate in an image query. To evaluate the benefits of the interface, it is compared to a traditional GUI application in which the users use a mouse to create a color layout sketch. Author Keywords Tangible user interface, image retrieval.

Composite images are synthesized from existing photographs by artists who make concept art, e.g., storyboards for movies or architectural planning. Current techniques allow an artist to fabricate such an image by digitally splicing parts of stock photographs. While these images serve mainly to "quickly" convey how a scene should look, their production is laborious. We propose a technique that allows a person to design a new photograph with substantially less effort. This paper presents a method that generates a composite image when a user types in nouns, such as "boat" and "sand." The artist can optionally design an intended image by specifying other constraints. Our algorithm formulates the constraints as queries to search an automatically annotated image database. The desired photograph, not a collage, is then synthesized using graph-cut optimization, optionally allowing for further user interaction to edit or choose among alternative generated photos. An implementation of our approach, shown in the associated video, demonstrates our contributions of (1) a method for creating specific images with minimal human effort, and (2) a combined algorithm for automatically building an image library with semantic annotations from any photo collection.