MAX-MARGIN MIN-ENTROPY MODELS

Abstract : We propose a new family of latent variable models called max-margin min-entropy (M3E) models, which define a distribution over the output and the hidden variables conditioned on the input. Given an input, an M3E model predicts the output with the smallest corresponding Renyi entropy of generalized distribution. This is equivalent to minimizing a score that consists of two terms: (i) the negative log-likelihood of the output, ensuring that the output has a high probability; and (ii) a measure of uncertainty over the distribution of the hidden variables conditioned on the input and the output, ensuring that there is little confusion in the values of the hidden variables. Given a training dataset, the parameters of an M3E model are learned by maximizing the margin between the Renyi entropies of the ground-truth output and all other incorrect outputs. Training an M3E can be viewed as minimizing an upper bound on a user-defined loss, and includes, as a special case, the latent support vector machine framework. We demonstrate the efficacy of M3E models on two standard machine learning applications, discriminative motif finding and image classification, using publicly available datase
Type de document :
Communication dans un congrès
AISTATS, Apr 2012, La Palma, Spain. 2012
Liste complète des métadonnées

Littérature citée [23 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-00773602
Contributeur : M. Pawan Kumar <>
Soumis le : lundi 14 janvier 2013 - 13:36:26
Dernière modification le : vendredi 12 janvier 2018 - 11:23:53
Document(s) archivé(s) le : lundi 15 avril 2013 - 04:00:36

Fichier

MKPGK-AISTATS2012.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-00773602, version 1

Collections

Citation

Kevin Miller, M. Pawan Kumar, Ben Packer, Danny Goodman, Daphne Koller. MAX-MARGIN MIN-ENTROPY MODELS. AISTATS, Apr 2012, La Palma, Spain. 2012. 〈hal-00773602〉

Partager

Métriques

Consultations de la notice

374

Téléchargements de fichiers

225