This paper presents a multi-classifier system using classifiers based on two different approaches. A stochastic model using Markov Random Field is combined with different kind of neural networks by several fusing rules. It has been proved that the combination of different classifiers can lead to improve the global recognition rate. We propose to compare different fusing rules in a framework composed of classifiers with high accuracies. We show that even there still remains a complementarity between classifiers, even from the same approach, that improves the global recognition rate. The combinations have been tested on handwritten digits. The overall recognition rate has reached 99.03\% without using any rejection criteria.