Verifiable Conditions for the Irreducibility and Aperiodicity of Markov Chains by Analyzing Underlying Deterministic Models

Abstract : We consider Markov chains that obey the following general non-linear state space model: $\Phi_{k+1} = F(\Phi_k, \alpha(\Phi_k, U_{k+1}))$ where the function $F$ is $C^1$ while $\alpha$ is typically discontinuous and $\{U_k: k \in \mathbb{Z}_{>0} \}$ is an independent and identically distributed process. We assume that for all $x$, the random variable $\alpha(x, U_1)$ admits a density $p_x$ such that $(x, w) \mapsto p_x(w)$ is lower semi-continuous. We generalize and extend previous results that connect properties of the underlying deterministic control model to provide conditions for the chain to be $\varphi$-irreducible and aperiodic. By building on those results, we show that if a rank condition on the controllability matrix is satisfied for all $x$, there is equivalence between the existence of a globally attracting state for the control model and $\varphi$-irreducibility of the Markov chain. Additionally, under the same rank condition on the controllability matrix, we prove that there is equivalence between the existence of a steadily attracting state and the $\varphi$-irreducibility and aperiodicity of the chain. The notion of steadily attracting state is new. We additionally derive practical conditions by showing that the rank condition on the controllability matrix needs to be verified only at a globally attracting state (resp.\ steadily attracting state) for the chain to be a $\varphi$-irreducible T-chain (resp.\ $\varphi$-irreducible aperiodic T-chain). Those results hold under considerably weaker assumptions on the model than previous ones that would require $(x,u) \mapsto F(x,\alpha(x,u))$ to be $C^\infty$ (while it can be discontinuous here). Additionally the establishment of a \emph{necessary and sufficient} condition for the $\varphi$-irreducibility and aperiodicity without a structural assumption on the control set is novel---even for Markov chains where $(x,u) \mapsto F(x,\alpha(x,u))$ is $C^\infty$. We illustrate that the conditions are easy to verify on a non-trivial and non-artificial example of Markov chain arising in the context of adaptive stochastic search algorithms to optimize continuous functions in a black-box scenario.
Type de document :
Article dans une revue
Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, In press
Liste complète des métadonnées

Littérature citée [17 références]  Voir  Masquer  Télécharger

https://hal.inria.fr/hal-01222222
Contributeur : Alexandre Chotard <>
Soumis le : jeudi 23 novembre 2017 - 10:23:16
Dernière modification le : vendredi 5 octobre 2018 - 09:59:57

Fichier

bernoulli-revision1.pdf
Fichiers produits par l'(les) auteur(s)

Identifiants

  • HAL Id : hal-01222222, version 2
  • ARXIV : 1508.01644

Citation

Alexandre Chotard, Anne Auger. Verifiable Conditions for the Irreducibility and Aperiodicity of Markov Chains by Analyzing Underlying Deterministic Models. Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, In press. 〈hal-01222222v2〉

Partager

Métriques

Consultations de la notice

180

Téléchargements de fichiers

46