Abstract: Mixtures of Probabilistic Principal Component Analyzers can be used to model data that lies on or near a low dimensional manifold in a high dimensional observation space, in effect tiling the manifold with local linear (Gaussian) patches. In order to exploit the low dimensional structure of the data manifold, the patches need to be localized and oriented in a low dimensional space, so that local coordinates on the patches can be mapped to global low dimensional coordinates. As shown by [Roweis et al., 2002], this problem can be expressed as a penalized likelihood optimization problem. We show that a restricted form of the Mixtures of Probabilistic Principal Component Analyzers model allows for an efficient EM-style algorithm. The Procrustes Rotation, a technique to match point configurations, turns out to give the optimal orientation of the patches in the global space. We also show how we can initialize the mappings from the patches to the global coordinates by learning a non-penalized density model first. Some experimental results are provided to illustrate the method.