Mathematics desk
< February 4	<< Jan \| February \| Mar >>	February 6 >

Welcome to the Wikipedia Mathematics Reference Desk Archives
The page you are currently viewing is an archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

February 5

Hessian Matrix Meaning

Let $f:\mathbb {R} ^{n}\to \mathbb {R}$ be a smooth function. Let $x\in \mathbb {R} ^{n}$ , such that the gradient of f at x is zero. Let H be the Hessian matrix of f at the point x. Let V be the vector space spanned by the eigenvectors corresponding to negative eigenvalues of H. Let $y\in V$ . Then, f(x)>f(y)? or maybe f(x)>f(x+y)?

In other words, does negative eigenvalue imply maximum point at the direction of the corresponding eigenvector, or maybe this is a maximum in another direction, and not in the direction of the eigenvector? עברית (talk) 06:45, 5 February 2016 (UTC)[reply]

See Morse lemma. Sławomir
Biały 12:23, 5 February 2016 (UTC)[reply]

(edit conflict)You're kind of circling around the second derivative test for functions of several variables. The Taylor expansion for f at x is

f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+\mathbf {y} ^{\mathrm {T} }\mathrm {D} f(\mathbf {x} )+{\frac {1}{2!}}\mathbf {y} ^{\mathrm {T} }\mathrm {D} ^{2}f(\mathbf {x} )\mathbf {y} +\cdots

where Df is the gradient and D²f is the Hessian. In this case the gradient is 0 at x so this reduces to

f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+{\frac {1}{2!}}\mathbf {y} ^{\mathrm {T} }\mathrm {D} ^{2}f(\mathbf {x} )\mathbf {y} +\cdots .

Let e be an eigenvector with eigenvalue λ, and wlog take e to be length 1. If y = te, then

f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+{\frac {1}{2!}}\lambda t^{2}+\cdots

so f has a local minimum or maximum along the line parallel to e though x, depending on whether λ is positive or negative. If e, f ... are several linearly independent eigenvectors, with eigenvalues λ, μ, ... , and y = te + uf + ... , then

f(\mathbf {x} +\mathbf {y} )\approx f(\mathbf {x} )+{\frac {1}{2!}}(\lambda t^{2}+\mu u^{2}+\cdots )+\cdots

so f has a local minimum or maximum in the relevant space though x provided λ, μ, ... have the same sign. (The eigenvectors may be taken to be orthogonal since D²f is symmetric.) Note, this is only valid for y sufficiently small, otherwise the higher order terms in the Taylor series become significant and the approximation is no longer valid. --RDBury (talk) 12:46, 5 February 2016 (UTC)[reply]

Oh, great! Thank you! :) עברית (talk) 08:38, 6 February 2016 (UTC)[reply]