Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera

Abstract : This paper presents a mirror-like augmented reality (AR) system to display the internal anatomy of a user. Using a single Microsoft V2.0 Kinect, we animate in real-time a user-specific internal anatomy according to the user’s motion and we superimpose it onto the user’s color map. The user can visualize his anatomy moving as if he was able to look inside his own body in real-time. A new calibration procedure to set up and attach a user-specific anatomy to the Kinect body tracking skeleton is introduced. At calibration time, the bone lengths are estimated using a set of poses. By using Kinect data as input, the practical limitation of skin correspondance in prior work is overcome. The generic 3D anatomical model is attached to the internal anatomy registration skeleton, and warped on the depth image using a novel elastic deformer, subject to a closest-point registration force and anatomical constraints. The noise in Kinect outputs precludes any realistic human display. Therefore, a novel filter to reconstruct plausible motions based on fixed length bones as well as realistic angular degrees of freedom (DOFs) and limits is introduced to enforce anatomical plausibility. Anatomical constraints applied to the Kinect body tracking skeleton joints are used to maximize the physical plausibility of the anatomy motion, while minimizing the distance to the raw data. At run-time, a simulation loop is used to attract the bones towards the raw data, and skinning shaders efficiently drag the resulting anatomy to the user’s tracked motion. Our user-specific internal anatomy model is validated by comparing the skeleton with segmented MRI images. A user study is established to evaluate the believability of the animated anatomy.
Document type :
Conference papers
Complete list of metadatas

Cited literature [39 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01366704
Contributor : Imagine Team <>
Submitted on : Friday, September 16, 2016 - 11:34:14 AM
Last modification on : Wednesday, April 3, 2019 - 1:15:47 AM
Long-term archiving on : Saturday, December 17, 2016 - 1:20:03 PM

Identifiers

Relations

Citation

Armelle Bauer, Ali-Hamadi Dicko, François Faure, Olivier Palombi, Jocelyne Troccaz. Anatomical Mirroring: Real-time User-specific Anatomy in Motion Using a Commodity Depth Camera. ACM SIGGRAPH Conference on Motion in Games, Oct 2016, San Francisco, United States. pp.113-122, ⟨10.1145/2994258.2994259⟩. ⟨hal-01366704⟩

Share

Metrics

Record views

1333

Files downloads

884