Object tracking with freely moving cameras is an open issue, since background information cannot be exploited for foreground segmentation, and plain feature tracking is not robust enough for target tracking, due to occlusions, distractors and object deformations. In order to deal with such challenging conditions a traditional approach, based on Camshift-like color-based features, is augmented by introducing a structural model of the object to be tracked incorporating previous knowledge about the spatial relations between the parts. Hence, an attributed graph is built on top of the features extracted from each frame and a graph matching technique based on Dominant Set clustering is used to find the optimal match with the model. Pixel-wise and object-wise comparison with other tracking techniques with respect to manually-obtained ground truth are presented.