We present an original method for tracking, in a monocular image sequence, complex objects which can be approximately modeled by a polyhedral shape. It considers two steps of global transformation, the first one operating in the 2D space and the second one in the 3D space. The first step is able to handle large displacements of the object projection in the image. In involves a 2D motion model estimated by a robust statistical method. Then, we refine the localization of the object silhouette by evaluating the 3D parameters related to the object pose by the iteratively minimizing a nonlinear cost function. It aims at moving the projection of the contours of the object CAD model to the spatial intensity gradients in the image. The proposed tracking method is real-time while being reliable and robust requirements. Real tracking experiments and results embedded in a visual servoing positioning task are reported.