Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera

Abstract : This paper presents a mirror-like augmented reality (AR) system to display the internal anatomy of a user. Using a single Microsoft V2.0 Kinect, we animate in real-time a user-specific internal anatomy according to the user’s motion and we superimpose it onto the user’s color map. The user can visualize his anatomy moving as if he was able to look inside his own body in real-time. A new calibration procedure to set up and attach a user-specific anatomy to the Kinect body tracking skeleton is introduced. At calibration time, the bone lengths are estimated using a set of poses. By using Kinect data as input, the practical limitation of skin correspondance in prior work is overcome. The generic 3D anatomical model is attached to the internal anatomy registration skeleton, and warped on the depth image using a novel elastic deformer, subject to a closest-point registration force and anatomical constraints. The noise in Kinect outputs precludes any realistic human display. Therefore, a novel filter to reconstruct plausible motions based on fixed length bones as well as realistic angular degrees of freedom (DOFs) and limits is introduced to enforce anatomical plausibility. Anatomical constraints applied to the Kinect body tracking skeleton joints are used to maximize the physical plausibility of the anatomy motion, while minimizing the distance to the raw data. At run-time, a simulation loop is used to attract the bones towards the raw data, and skinning shaders efficiently drag the resulting anatomy to the user’s tracked motion. Our user-specific internal anatomy model is validated by comparing the skeleton with segmented MRI images. A user study is established to evaluate the believability of the animated anatomy.
Type de document :
Communication dans un congrès
ACM SIGGRAPH Conference on Motion in Games, Oct 2016, San Francisco, United States. ACM, pp.113-122, <10.1145/2994258.2994259>
Liste complète des métadonnées


https://hal.archives-ouvertes.fr/hal-01366704
Contributeur : Imagine Team <>
Soumis le : vendredi 16 septembre 2016 - 11:34:14
Dernière modification le : mercredi 15 mars 2017 - 12:12:12
Document(s) archivé(s) le : samedi 17 décembre 2016 - 13:20:03

Identifiants

Relations

Citation

Armelle Bauer, Ali-Hamadi Dicko, François Faure, Olivier Palombi, Jocelyne Troccaz. Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera. ACM SIGGRAPH Conference on Motion in Games, Oct 2016, San Francisco, United States. ACM, pp.113-122, <10.1145/2994258.2994259>. <hal-01366704>

Partager

Métriques

Consultations de
la notice

536

Téléchargements du document

433