In this paper, we propose a new statistical framework for modeling and extracting 2D moving deformable objects from image sequences. The object representation relies on a hierarchical description of the deformation applied to a computed template. Global deformations are modeled using a Karhunen Loeve expansion of the distorsions observed on a representative population. Local deformations are modeled using (first-order) Markov processes. The statistical hierarchical model is used to represent the a priori structure of the shapes to be extracted from the image sequence. The optimal bayesian estimate of the global and local deformations is obtained by minimizing a global objective function depending on the global deformation parameters and on the local random deformation process. A partial optimal solution is estimated by stochastic and deterministic relaxation procedures. The use of global optimization algorithms yields robust and reliable segmentations in adverse situations such as low signal-to-noise ratio, non-gaussian noise or occlusions. This procedure also leads to solutions which do not depend on the initial configuration of the model. The technique is demonstrated on synthetic as well as on real-world image sequences showing moving hands with partial occlusions.