Cone of view camera model using conformal geometric algebra for classic and panoramic image sensors

Abstract : Camera calibration is a necessary step in 3D computer vision in order to extract metric information from 2D images. Calibration has been defined as the non parametric association of a projection ray in 3D to every pixel in an image. It is normally neglected that pixels have a finite surface that can be approximated by a cone of view that has the usual ray of view of the pixel as a directrix axis. If this pixels' physical topology can be easily neglected in the case of perspective cameras, it is an absolute necessity to consider it in the case of variant scale cameras such as foveolar or catadioptric omnidirectional sensors which are nowadays widely used in robotics. This paper presents a general model to geometrically describe cameras whether they have a constant or variant scale resolution by introducing the new idea of using pixel-cones to model the field of view of cameras rather than the usual line-rays. The paper presents the general formulation using twists of conformal geometric algebra to express cones and their intersections in an easy and elegant manner, and without which the use of cones would be too binding. The paper will also introduce an experimental method to determine pixels-cones for any geometric type of camera. Experimental results will be shown in the case of perspective and omnidirectional catadioptric cameras.