This paper describes a number of geometric tools that can be easily implemented in order to obtain a very robust visual servoing platform. Most often, visual servoing algorithms are either based on calibrated systems or dispose of enough information to self-calibrate. We show that this abundance of information can be exploited in the case of stereo servoing as to offer a solution for some problems related to signal loss such as temporary occlusions, specularities or CAD tracking difficulties. The presented servoing algorithm, can cope with large image perturbations and is able to control a robot through a non modeled set of reference points.