In this paper a fast data driven learning-corpus building algorithm (FDDLCB) is proposed. The generic technique allows to build dynamically a representative and compact learning corpus for a connectionist model. The constructed dataset contains just a reduced number of patterns but sufficiently descriptive to characterize the different classes which should be separated. The method is based on a double least mean squares (LMS) error minimization mechanism trying to find the optimal boundaries of the different pattern classes. In the classical learning process the LMS is serving to minimize the error during the learning and this process is improved with a second one, as the new samples selection is also based on the idea to minimize the recognition error. Reinforcing the class boundaries where the recognition fails let us achieve a rapid and good generalization without any loss of occuracy. A modified version of the algorithm will be also presented. The experiments were performed on MNIST (Modified NIST) separated digit dataset. The encouraging result (98.51%) using just 1.85% of the available patterns from the original training dataset is comparable even with the state of the art techniques.