Skip to Main content Skip to Navigation
New interface
Conference papers

Understanding Everyday Hands in Action from RGB-D Images

Gregory Rogez 1 James Steven Supancic 2 Deva Ramanan 3 
1 LEAR - Learning and recognition in vision
Inria Grenoble - Rhône-Alpes, LJK - Laboratoire Jean Kuntzmann, Grenoble INP - Institut polytechnique de Grenoble - Grenoble Institute of Technology
Abstract : We analyze functional manipulations of handheld objects, formalizing the problem as one of fine-grained grasp classification. To do so, we make use of a recently developed fine-grained taxonomy of human-object grasps. We introduce a large dataset of 12000 RGB-D images covering 71 everyday grasps in natural interactions. Our dataset is different from past work (typically addressed from a robotics perspective) in terms of its scale, diversity, and combination of RGB and depth data. From a computer-vision perspective , our dataset allows for exploration of contact and force prediction (crucial concepts in functional grasp analysis) from perceptual cues. We present extensive experimental results with state-of-the-art baselines, illustrating the role of segmentation, object context, and 3D-understanding in functional grasp analysis. We demonstrate a near 2X improvement over prior work and a naive deep baseline, while pointing out important directions for improvement.
Document type :
Conference papers
Complete list of metadata

Cited literature [42 references]  Display  Hide  Download
Contributor : Gregory Rogez Connect in order to contact the contributor
Submitted on : Wednesday, December 2, 2015 - 3:17:58 PM
Last modification on : Friday, November 18, 2022 - 9:28:18 AM
Long-term archiving on: : Friday, April 28, 2017 - 11:22:47 PM


Files produced by the author(s)




Gregory Rogez, James Steven Supancic, Deva Ramanan. Understanding Everyday Hands in Action from RGB-D Images. ICCV - IEEE International Conference on Computer Vision, Dec 2015, Santiago, Chile. pp.3889-3897, ⟨10.1109/ICCV.2015.443⟩. ⟨hal-01237011⟩



Record views


Files downloads