This paper present an approach for detection, labelling and tracking multiple objects through both temporally and spatially significant occlusions. The proposed method builds on the idea of object permanence to reason about occlusion. To this end, tracking is performed at both the region level and the object level. At the region level, a particle filter is used to search for optimal region tracks. This limits the scope of object trajectories. At the object level, each object is located based on adaptive appearance models, spatial distributions and inter-occlusion relationships. Region covariance matrices are used to model objects appearance and the dissimilarity between region covariance matrices is used as a new measurement for the particles weight. The regions covariance matrices are updated using a novel approach in a Riemannian space. The proposed architecture is capable of tracking multiple objects even in the presence of periods of full occlusions using a simple and efficient solution for group handling and occlusion reasoning. The results shows the effectiveness of the approach hereby proposed.