Abstract : We present a variational Expectation-Maximization algorithm to learn proba- bilistic mixture models. The algorithm is similar to Kohonen's Self-Organizing Map algorithm and can be applied on any mixture model for which we can find a standard Expectation Maximization algorithm. We maximize the variational free- energy which sums data log-likelihood and Kullback-Leibler divergence between the neighborhood function and the posterior distribution on the components, given data. We illustrate the algorithm with an application on word clustering.
