Kernel Approximation Methods for Speech Recognition

Abstract : We study large-scale kernel methods for acoustic modeling in speech recognition and compare their performance to deep neural networks (DNNs). We perform experiments on four speech recognition datasets, including the TIMIT and Broadcast News benchmark tasks, and compare these two types of models on frame-level performance metrics (accuracy, cross-entropy), as well as on recognition metrics (word/character error rate). In order to scale kernel methods to these large datasets, we use the random Fourier feature method of Rahimi and Recht [2007]. We propose two novel techniques for improving the performance of kernel acoustic models. First, in order to reduce the number of random features required by kernel models, we propose a simple but effective method for feature selection. The method is able to explore a large number of non-linear features while maintaining a compact model more efficiently than existing approaches. Second, we present a number of frame-level metrics which correlate very strongly with recognition performance when computed on the heldout set; we take advantage of these correlations by monitoring these metrics during training in order to decide when to stop learning. This technique can noticeably improve the recognition performance of both DNN and kernel models, while narrowing the gap between them. Additionally, we show that the linear bottleneck method of Sainath et al. [2013a] improves the performance of our kernel models significantly, in addition to speeding up training and making the models more compact. Together, these three methods dramatically improve the performance of kernel acoustic models, making their performance comparable to DNNs on the tasks we explored.
Type de document :
Rapport
[Research Report] INRIA Lille. 2017, pp.1-31
Liste complète des métadonnées

Littérature citée [68 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01665417
Contributeur : Aurélien Bellet <>
Soumis le : vendredi 15 décembre 2017 - 19:28:22
Dernière modification le : vendredi 13 avril 2018 - 01:28:51

Fichier

main_arxiv.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01665417, version 1
  • ARXIV : 1701.03577

Collections

Citation

Avner May, Alireza Bagheri Garakani, Zhiyun Lu, Dong Guo, Kuan Liu, et al.. Kernel Approximation Methods for Speech Recognition. [Research Report] INRIA Lille. 2017, pp.1-31. 〈hal-01665417〉

Partager

Métriques

Consultations de la notice

97

Téléchargements de fichiers

33