Non-negative Spectral Learning for Linear Sequential Systems

Abstract : Method of moments (MoM) has recently become an appealing alternative to standard iterative approaches like Expectation Maximization (EM) to learn latent variable models. In addition, MoM-based algorithms come with global convergence guarantees in the form of finite sample bounds. However, given enough computation time, by using restarts and heuristics to avoid local optima, iterative approaches often achieve better performance. We believe that this performance gap is in part due to the fact that MoM-based algorithms can output negative probabilities. By constraining the search space, we propose a non-negative spectral algorithm (NNSpectral) avoiding computing negative probabilities by design. NNSpectral is compared to other MoM-based algorithms and EM on synthetic problems of the PAutomaC challenge. Not only, NNSpectral outperforms other MoM-based algorithms, but also, achieves very competitive results in comparison to EM.
Complete list of metadatas

Cited literature [17 references]  Display  Hide  Download

https://hal.inria.fr/hal-01225838
Contributor : Olivier Pietquin <>
Submitted on : Friday, November 6, 2015 - 6:26:40 PM
Last modification on : Thursday, April 4, 2019 - 10:18:05 AM
Long-term archiving on : Monday, February 8, 2016 - 1:02:10 PM

File

ICONIP_2015_HGCEOP.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-01225838, version 1

Citation

Hadrien Glaude, Cyrille Enderli, Olivier Pietquin. Non-negative Spectral Learning for Linear Sequential Systems. 22nd International Conference on Neural Information Processing (ICONIP2015), Nov 2015, Istanbul, Turkey. ⟨hal-01225838⟩

Share

Metrics

Record views

249

Files downloads

219