High Performance Convolutional Neural Networks for Document Processing

Abstract : Convolutional neural networks (CNNs) are well known for producing state-of-the-art recognizers for document processing [1]. However, they can be difficult to implement and are usually slower than traditional multi-layer perceptrons (MLPs). We present three novel approaches to speeding up CNNs: a) unrolling convolution, b) using BLAS (basic linear algebra subroutines), and c) using GPUs (graphic processing units). Unrolled convolution converts the processing in each convolutional layer (both forward-propagation and back-propagation) into a matrix-matrix product. The matrix-matrix product representation of CNNs makes their implementation as easy as MLPs. BLAS is used to efficiently compute matrix products on the CPU. We also present a pixel shader based GPU implementation of CNNs. Results on character recognition problems indicate that unrolled convolution with BLAS produces a dramatic 2.4X−3.0X speedup. The GPU implementation is even faster and produces a 3.1X−4.1X speedup.
Complete list of metadatas

Cited literature [5 references]  Display  Hide  Download

https://hal.inria.fr/inria-00112631
Contributor : Anne Jaigu <>
Submitted on : Thursday, November 9, 2006 - 1:47:33 PM
Last modification on : Thursday, November 9, 2006 - 4:43:26 PM
Long-term archiving on : Tuesday, April 6, 2010 - 9:59:52 PM

Identifiers

  • HAL Id : inria-00112631, version 1

Collections

Citation

Kumar Chellapilla, Sidd Puri, Patrice Simard. High Performance Convolutional Neural Networks for Document Processing. Tenth International Workshop on Frontiers in Handwriting Recognition, Université de Rennes 1, Oct 2006, La Baule (France). ⟨inria-00112631⟩

Share

Metrics

Record views

4315

Files downloads

22633