Hybrid tracking algorithms for planar and non-planar structures subject to illumination changes

Abstract : Augmented Reality (AR) aims to fuse a virtual world and a real one in an image stream. When considering only a vision sensor, it relies on registration techniques that have to be accurate and fast enough for on-line augmentation. This paper proposes a real-time, robust and efficient 3D model-based tracking algorithm for a 'video see through' monocular vision system. A virtual visual servoing approach is used to estimate the pose between the camera and the object. The integration of texture information in the classical non-linear edge-based pose computation provides a more reliable tracker. Several illumination models have been considered and compared to better deal with the illumination change in the scene. Robustness is enforced by integrating a M-estimator into the virtual visual control law via an iteratively re-weighted least squares implementation. The method presented in this paper has been validated on several video sequences for augmented reality applications. Results show the method to be robust to large motions and textured environments.