Background: Nowadays, cardiovascular diseases (CVD) remain the main cause of death worldwide. A heart sound signal or phonocardiogram (PCG) is the most simple, economical and non-invasive tool to detect CVDs. Advances in technology and signal processing allow the design of computer-aided systems for heart illnesses detection from PCG signals. Purpose: The paper proposes a pipeline and benchmark for binary heart sounds classification. The features extraction architecture is focused on the use of Matching Pursuit time-frequency decomposition using Gabor dictionaries and the Linear Predictive Coding method of a residual. We compare seven classifiers with two different approaches: feature averaging and cycle averaging. Methods: We test our proposal on the PhysioNet/CinC challenge 2016 database, which comprises a wide variety of heart sounds recorded from patients with normal and different pathological heart conditions. We conduct a 10-fold stratified cross-validation method to evaluate the performance of different classification algorithms. The feature sets were also tested when using an oversampling method for balancing. Results: The benchmark identified systems showing a satisfying performance in terms of accuracy, sensitivity, and Matthews correlation coefficient. Results can be improved when using feature averaging and an oversampling strategy.