Nowadays, most weather forecasting centers produce ensemble forecasts. Ensemble forecasts provide information about probability distribution of the weather variables. They give a more complete description of the atmosphere than a unique run of the meteorological model. However, they may suffer from bias and under/over dispersion errors that need to be corrected. These distribution errors may depend on weather regimes. In this paper, we propose various extensions of the Gaussian mixture model and its associated inference tools for ensemble data sets. The proposed models are then used to identify clusters which correspond to different types of distribution errors. Finally, a standard calibration method known as Non homogeneous Gaussian Regression (NGR) is applied cluster by cluster in order to correct ensemble forecast distributions. It is shown that the proposed methodology is effective, interpretable and easy to use. The clustering algorithms are illustrated on simulated and real data. The calibration method is applied to real data of temperature and wind medium range forecast for 3 stations in France.