Robustness and accuracy are major issues in real-time object tracking in image sequences. This paper describes a reliable tracking for markerless objects based on the fusion of visual cues and on the estimation of a 2D transformation. The parameters of this transformation are estimated using a non-linear minimization of a unique criterion that integrates information both on the texture and the edges of the tracked object. The proposed tracker is then more robust and succeeds in conditions where methods based on a single cue fail. The efficiency and the robustness of the proposed method are tested on image sequences as well as during an image-based visual servoing experiment.