非线性最小二乘法Levenberg-Marquardt method

Levenberg-Marquardt Method(麦夸尔特法)

Levenberg-Marquardt is a popular alternative to the Gauss-Newton method of finding the minimum of a

function that is a sum of squares of nonlinear functions,

Let the Jacobian of be denoted , then the Levenberg-Marquardt method searches in the

direction given by the solution to the equations

where are nonnegative scalars and is the identity matrix. The method has the nice property that, for

some scalar related to , the vector is the solution of the constrained subproblem of minimizing

subject to (Gill et al. 1981, p. 136).

The method is used by the command FindMinimum[f, x, x0] when given the Method -> Levenberg Marquardt option.

SEE A LSO:Minimum, Optimization

REFERENCES:

Bates, D. M. and Watts, D. G. N onlinear Regr ession and Its Applications. New York: Wiley, 1988.

Gill, P. R.; Murray, W.; and Wright, M. H. "The Levenberg-Marquardt Method." §4.7.3 in Practical Optim ization. London: Academic Press, pp. 136-137, 1981.

Levenberg, K. "A Method for the Solution of Certain Problems in Least Squares." Quart. Appl. Math.2, 164-168, 1944. Marquardt, D. "An Algor ithm for Least-Squares Estimation of Nonlinear Parameters." SIAM J. Appl. Math.11, 431-441, 1963.

Levenberg–Marquardt algorithm

From Wikipedia, the free encyclopedia

Jump to: navigation, search

In mathematics and computing, the Levenberg–Marquardt algorithm (LMA)[1] provides a numerical solution to the problem of minimizing a function, generally nonlinear, over a space of parameters of the function. These minimization problems arise especially in least squares curve fitting and nonlinear programming.

The LMA interpolates between the Gauss–Newton algorithm (GNA) and the method of gradient descent. The LMA is more robust than the GNA, which means that in many cases it finds a solution even if it starts very far off the final minimum. For well-behaved functions and reasonable starting parameters, the LMA tends to be a bit slower than the GNA. LMA can also be viewed as Gauss–Newton using a trust region approach.

The LMA is a very popular curve-fitting algorithm used in many software applications for solving generic curve-fitting problems. However, the LMA finds only a local minimum, not a global minimum.

Contents

[hide]

∙ 1 Caveat Emptor

∙ 2 The problem

∙ 3 The solution

o 3.1 Choice of damping parameter

∙ 4 Example

∙ 5 Notes

∙ 6 See also

∙7 References

∙8 External links

o8.1 Descriptions

o8.2 Implementations

[edit] Caveat Emptor