Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Robust and Efficient Optimization Using a Marquardt-Levenberg Algorithm with R Package marqLevAlg

Abstract : Optimization is an essential task in many computational problems. In statistical modelling for instance, in the absence of analytical solution, maximum likelihood estimators are often retrieved using iterative optimization algorithms. R software already includes a variety of optimizers from general-purpose optimization algorithms to more specific ones. Among Newton-like methods which have good convergence properties, the Marquardt-Levenberg algorithm (MLA) provides a particularly robust algorithm for solving optimization problems. Newton-like methods generally have two major limitations: (i) convergence criteria that are a little too loose, and do not ensure convergence towards a maximum, (ii) a calculation time that is often too long, which makes them unusable in complex problems. We propose in the marqLevAlg package an efficient and general implementation of a modified MLA combined with strict convergence criteria and parallel computations. Convergence to saddle points is avoided by using the relative distance to minimum/maximum criterion (RDM) in addition to the stability of the parameters and of the objective function. RDM exploits the first and second derivatives to compute the distance to a true local maximum. The independent multiple evaluations of the objective function at each iteration used for computing either first or second derivatives are called in parallel to allow a theoretical speed up to the square of the number of parameters. We show through the estimation of 7 relatively complex statistical models how parallel implementation can largely reduce computational time. We also show through the estimation of the same model using 3 different algorithms (BFGS of optim routine, an E-M, and MLA) the superior efficiency of MLA to correctly and consistently reach the maximum.
Document type :
Preprints, Working Papers, ...
Complete list of metadata

https://hal.inria.fr/hal-03100489
Contributor : Boris Hejblum Connect in order to contact the contributor
Submitted on : Thursday, January 7, 2021 - 7:30:04 PM
Last modification on : Wednesday, November 3, 2021 - 4:16:12 AM
Long-term archiving on: : Thursday, April 8, 2021 - 7:56:09 PM

File

mlaSubmit.pdf
Files produced by the author(s)

Identifiers

  • HAL Id : hal-03100489, version 1
  • ARXIV : 2009.03840

Collections

Citation

Viviane Philipps, Boris P. Hejblum, Mélanie Prague, Daniel Commenges, Cécile Proust-Lima. Robust and Efficient Optimization Using a Marquardt-Levenberg Algorithm with R Package marqLevAlg. 2020. ⟨hal-03100489⟩

Share

Metrics

Record views

61

Files downloads

278