Skip to Main content Skip to Navigation
New interface
Conference papers

Mixed Reality Crowds: Online Inserting Virtual Characters into Dynamic Video Scenes

Yijiang Zhang 1, 2 Julien Pettré 1 Jan Ondrej 1 Qin Xueying 2 Qunsheng Peng 2 Stéphane Donikian 1 
1 MIMETIC - Analysis-Synthesis Approach for Virtual Human Simulation
UR2 - Université de Rennes 2, Inria Rennes – Bretagne Atlantique , IRISA-D6 - MEDIA ET INTERACTIONS
Abstract : The seamless integration of virtual characters into dynamic scenes captured by video is a challenging problem. In order to achieve consistent composite results, both the virtual and real characters must share the same geometrical space and their interactions must adhere to the physical coherence criteria. One essential question is how to detect the motion of real objects - such as real characters moving in the video - and how to steer virtual characters accordingly to avoid unrealistic collisions. We propose an online solution. First, by analysis of the input video, the motion states of the real objects are recovered into a common world 3D coordinate system. Meanwhile, a simplified accuracy measurement is defined to represent the confidence of the motion estimate. Then, under the constraints imposed by the real dynamic objects, the motion of virtual characters are accommodated by a uniform steering model. The final step is to merge virtual objects back the real video scene by taking into account visibility and occlusion constraints between real foreground objects and virtual ones: depth is captured into alpha maps the content of which is refined. Specifically, the estimate is used to reduce the ambiguity caused by occlusions between real objects in the original video. Several examples demonstrate the efficiency of the proposed algorithm.
Document type :
Conference papers
Complete list of metadata
Contributor : Julien Pettré Connect in order to contact the contributor
Submitted on : Sunday, November 27, 2011 - 12:23:33 PM
Last modification on : Saturday, June 25, 2022 - 7:39:51 PM


  • HAL Id : hal-00645210, version 1


Yijiang Zhang, Julien Pettré, Jan Ondrej, Qin Xueying, Qunsheng Peng, et al.. Mixed Reality Crowds: Online Inserting Virtual Characters into Dynamic Video Scenes. CASA 2011, May 2011, Chengdu, China. ⟨hal-00645210⟩



Record views