In this paper we propose a novel method for knowledge-based segmentation. We adopt a point distribution graphical model formulation which encodes pose invariant shape priors through L1 sparse higher order cliques. Local shape deformation properties of the model can be captured and learned in an optimal manner from a training set using dual decomposition. These higher order shape terms are combined with conventional visual ones aiming at maximizing the posterior segmentation likelihood. The considered graphical model is optimized using dual decomposition and is used towards 2D (computer vision) and 3D object segmentation (medical imaging) with promising results.