The automated analysis of 3D medical images can improve both diagnosis and therapy significantly. This automation raises a number of new fascinating research problems in the fields of computer vision, graphics and robotics. In this paper, I propose a list of such problems after a review of the current major 3D imaging modalities, and a description of the related medical needs. I then present some of the past and current work done in our research group EPIDAURE at INRIA, on the following topics: segmentation of 3D images; 3D shape modelling; 3D rigid and nonrigid registration; 3D motion analysis; and 3D simulation of therapy. Most topics are discussed in a synthetic manner, and illustrated by results. Rigid matching is treated more thoroughly as an illustration of a transfer from computer vision towards 3D image processing. The later topics are illustrated by preliminary results, and a number of promising research tracks are suggested.