Skip to Main content Skip to Navigation
Conference papers

Data Fine-Pruning: A Simple Way to Accelerate Neural Network Training

Abstract : The training process of a neural network is the most time-consuming procedure before being deployed to applications. In this paper, we investigate the loss trend of the training data during the training process. We find that given a fixed set of hyper-parameters, pruning specific types of training data can reduce the time consumption of the training process while maintaining the accuracy of the neural network. We developed a data fine-pruning approach, which can monitor and analyse the loss trend of training instances at real-time, and based on the analysis results, temporarily pruned specific instances during the training process basing on the analysis. Furthermore, we formulate the time consumption reduced by applying our data fine-pruning approach. Extensive experiments with different neural networks are conducted to verify the effectiveness of our method. The experimental results show that applying the data fine-pruning approach can reduce the training time by around 14.29% while maintaining the accuracy of the neural network.
Document type :
Conference papers
Complete list of metadata

Cited literature [25 references]  Display  Hide  Download

https://hal.inria.fr/hal-02279554
Contributor : Hal Ifip <>
Submitted on : Thursday, September 5, 2019 - 1:31:18 PM
Last modification on : Thursday, September 5, 2019 - 1:35:33 PM
Long-term archiving on: : Thursday, February 6, 2020 - 3:30:27 AM

File

477597_1_En_10_Chapter.pdf
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Junyu Li, Ligang He, Shenyuan Ren, Rui Mao. Data Fine-Pruning: A Simple Way to Accelerate Neural Network Training. 15th IFIP International Conference on Network and Parallel Computing (NPC), Nov 2018, Muroran, Japan. pp.114-125, ⟨10.1007/978-3-030-05677-3_10⟩. ⟨hal-02279554⟩

Share

Metrics

Record views

70

Files downloads

23