Mixed Reality Crowds: Online Inserting Virtual Characters into Dynamic Video Scenes

Yijiang Zhang 1, 2 Julien Pettré 1 Jan Ondrej 1 Qin Xueying 2 Qunsheng Peng 2 Stéphane Donikian 1
1 MIMETIC - Analysis-Synthesis Approach for Virtual Human Simulation
IRISA-D6 - MEDIA ET INTERACTIONS, Inria Rennes – Bretagne Atlantique , UR2 - Université de Rennes 2
Abstract : The seamless integration of virtual characters into dynamic scenes captured by video is a challenging problem. In order to achieve consistent composite results, both the virtual and real characters must share the same geometrical space and their interactions must adhere to the physical coherence criteria. One essential question is how to detect the motion of real objects - such as real characters moving in the video - and how to steer virtual characters accordingly to avoid unrealistic collisions. We propose an online solution. First, by analysis of the input video, the motion states of the real objects are recovered into a common world 3D coordinate system. Meanwhile, a simplified accuracy measurement is defined to represent the confidence of the motion estimate. Then, under the constraints imposed by the real dynamic objects, the motion of virtual characters are accommodated by a uniform steering model. The final step is to merge virtual objects back the real video scene by taking into account visibility and occlusion constraints between real foreground objects and virtual ones: depth is captured into alpha maps the content of which is refined. Specifically, the estimate is used to reduce the ambiguity caused by occlusions between real objects in the original video. Several examples demonstrate the efficiency of the proposed algorithm.
Type de document :
Communication dans un congrès
CASA 2011, May 2011, Chengdu, China. 2011
Liste complète des métadonnées

Contributeur : Julien Pettré <>
Soumis le : dimanche 27 novembre 2011 - 12:23:33
Dernière modification le : jeudi 15 novembre 2018 - 11:57:49


  • HAL Id : hal-00645210, version 1


Yijiang Zhang, Julien Pettré, Jan Ondrej, Qin Xueying, Qunsheng Peng, et al.. Mixed Reality Crowds: Online Inserting Virtual Characters into Dynamic Video Scenes. CASA 2011, May 2011, Chengdu, China. 2011. 〈hal-00645210〉



Consultations de la notice