Skip to Main content Skip to Navigation
Conference papers

Channels’ Matching Algorithm for Mixture Models

Abstract : To solve the Maximum Mutual Information (MMI) and Maximum Likelihood (ML) for tests, estimations, and mixture models, it is found that we can obtain a new iterative algorithm by the Semantic Mutual Information (SMI) and R(G) function proposed by Chenguang Lu (1993) (where R(G) function is an extension of information rate distortion function R(D), G is the lower limit of the SMI, and R(G) represents the minimum R for given G). This paper focus on mixture models. The SMI is defined by the average log normalized likelihood. The likelihood function is produced from the truth function and the prior by the semantic Bayesian inference. A group of truth functions constitute a semantic channel. Letting the semantic channel and Shannon channel mutually match and iterate, we can obtain the Shannon channel that maximizes the MMI and the average log likelihood. Therefore, this iterative algorithm is called Channels’ Matching algorithm or the CM algorithm. It is proved that the relative entropy between the sampling distribution and predicted distribution may be equal to R − G. Hence, solving the maximum likelihood mixture model only needs minimizing R − G, without needing Jensen’s inequality. The convergence can be intuitively explained and proved by the R(G) function. Two iterative examples of mixture models (which are demonstrated in an excel file) show that the computation for the CM algorithm is simple. In most cases, the number of iterations for convergence (as the relative entropy <0.001 bit) is about 5. The CM algorithm is similar to the EM algorithm; however, the CM algorithm has better convergence and more potential applications.
Document type :
Conference papers
Complete list of metadata

Cited literature [18 references]  Display  Hide  Download

https://hal.inria.fr/hal-01820942
Contributor : Hal Ifip <>
Submitted on : Friday, June 22, 2018 - 10:44:53 AM
Last modification on : Friday, June 22, 2018 - 10:51:21 AM
Long-term archiving on: : Tuesday, September 25, 2018 - 12:26:04 AM

File

978-3-319-68121-4_35_Chapter.p...
Files produced by the author(s)

Licence


Distributed under a Creative Commons Attribution 4.0 International License

Identifiers

Citation

Chenguang Lu. Channels’ Matching Algorithm for Mixture Models. 2nd International Conference on Intelligence Science (ICIS), Oct 2017, Shanghai, China. pp.321-332, ⟨10.1007/978-3-319-68121-4_35⟩. ⟨hal-01820942⟩

Share

Metrics

Record views

136

Files downloads

68