Abstract

We present an approach which exploits the coupling between human actions and scene geometry. We investigate the use of human pose as a cue for single-view 3D scene understanding. Our method builds upon recent advances in still-image action recognition and pose estimation, to extract functional and geometric constraints about the scene from people detections. These constraints are then used to improve state-of-the-art single-view 3D scene understanding approaches. The proposed method is validated on a collection of single-viewpoint time-lapse image sequences as well as a dataset of still images of indoor scenes. We demonstrate that observing people performing different actions can significantly improve estimates of scene geometry and 3D layout. 1.

Citations

2275	Multiview view geometry in computer vision - Hartley, Zisserman - 2003
1234	The ecological approach to visual perception - Gibson - 1979
809	Histograms of oriented gradients for human detection - Dalal, Triggs - 2005
111	Geometric context from a single image - Hoiem, Efros, et al.
106	Putting Objects in Perspective - Hoiem, Efros, et al. - 2008
96	Reconstruction of articulated objects from point correspondences in a single image - Taylor
56	Poselets: Body part detectors trained using 3d human pose annotations - Bourdev, Malik - 2009
45	Modeling the world from internet photo collections - Snavely, Seitz, et al. - 2008
41	Pictorial Structures Revisited: People Detection and Articulated Pose Estimation - Andriluka, Roth, et al. - 2009
29	Modeling mutual context of object and human pose in humanobject interaction activities - Bangpeng, Fei-Fei - 2010
23	Recovering the spatial layout of cluttered rooms - Hedau, Hoiem, et al. - 2009
19	L.S.: Objects in action: An approach for combining action understanding and object perception - Gupta, Davis - 2007
18	P.R.S.: Bayesian autocalibration for surveillance - Krahnstoever, Mendonca - 2005
16	D.: Simultaneous visual recognition of manipulation actions and manipulated objects. In: ECCV - Kjellstrom, Romero, et al. - 2008
15	M.: Blocks world revisited: Image understanding using qualitative geometry and mechanics - Gupta, Efros, et al. - 2010
14	D.: Thinking inside the box: Using appearance models and context based on room geometry. In: ECCV - Hedau, Hoiem, et al. - 2010
13	M.: 3d occlusion inference from silhouette cues - Guan, Franco, et al.
13	Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces - Lee, Gupta, et al. - 2010
13	Articulated pose estimation using flexible mixture of parts - Yang, Ramanan - 2011
11	Discriminative models for static human-object interactions - Desai, Ramanan, et al. - 2010
10	D.: Discriminative learning with latent variables for cluttered indoor scene understanding,. In: ECCV - Wang, Gould, et al. - 2010
7	Learning to recognize objects in egocentric activities - Fathi, Ren, et al. - 2011
7	Weakly supervised learning of interactions between humans and objects - Prest, Schmid - 2011
6	M.: Learning effective human pose estimation from inaccurate annotation - Johnson, Everingham
6	G.: What can casual walkers tell us about the 3D scene - Rother, Patwardhan, et al. - 2007
6	R.: Unsupervised learning of functional categories in video scenes. In: ECCV - Turek, Hoogs, et al. - 2010
5	P.: Geometric image parsing in man-made environments - Barinova, Lempitsky, et al. - 2010
5	I.: Learning person-object interactions for action recognition in still images. In: NIPS. (2011) semantics from long-term observation of people 13 - Delaitre, Sivic, et al.
4	S.: Scene shape from texture of objects - Payet, Todorovic
4	K.: Sampling bedrooms - Pero, Guan, et al.
3	L.: Classifying actions and measuring action similarity by modeling the mutual context of objects and human poses - Yao, Khosla, et al.
2	W.Grimson; “Adaptive background mixture models for real time tracking - Staufer - 1999
1	From scene geometry to human workspace - Gupta, Satkin, et al.