User:LachlanA/Convergence proof techniques

Convergence proof techniques are canonical components of mathematical proofs that sequences or functions converge to a finite limit when the argument tends to infinity.

There are many types of series and modes of convergence requiring different techniques. Below are some of the more common examples. This article is intended as an introduction aimed to help practitioners explore appropriate techniques. The links below give details of necessary conditions and generalizations to more abstract settings. The convergence of series is already covered in the article on convergence tests.

Convergence in Rⁿ

It is common to want to prove convergence of a sequence $f:\mathbb {N} \rightarrow \mathbb {R} ^{n}$ or function $f:\mathbb {R} \rightarrow \mathbb {R} ^{n}$ , where $\mathbb {N}$ and $\mathbb {R}$ refer to the natural numbers and the real numbers, and convergence is with respect to the Euclidean norm, $||\cdot ||_{2}$ .

Useful approaches for this are as follows.

First principles

The analytic definition of convergence of $f$ to a limit $f_{\infty }$ is that for all $\epsilon$ there exists a $k_{0}$ such for all $k>k_{0}$ , $||f(k)-f_{\infty }||<\epsilon$ . The most basic proof technique is to find such a $k_{0}$ and prove the required inequality. If the value of $L$ is not known in advance, the techniques below may be useful.

Contraction mappings

In many cases, the function whose convergence is of interest has the form $f(k+1)=T(f(k))$ for some transformation $T$ . For example, $T$ could map $f(k)$ to $f(k+1)=Af(k)$ for some conformable matrix $A$ . Alternatively, $T$ may be an element-wise operation, such as replacing each element of $f(k)$ by the square root of its magnitude.

In such cases, if the problem satisfies the conditions of Banach fixed-point theorem (the domain is a non-empty complete metric space) then it is sufficient to prove that $||T(x)-T(y)||<k(x-y)||$ for some constant $|k|<1$ which is fixed for all $x$ and $y$ . Such a $T$ is called a contraction mapping.

Monotonicity (Lyapunov functions)

Every bounded monotonic sequence has a convergent subsequence. If it can be shown that all of the subsequences of $f$ have the same limit, such as by showing that there is a unique fixed point of the transformation $T$ , then the initial sequence must also converge to that limit.

This approach can also be applied to sequences that are not monotonic. Instead, it is possible to define a function $V:\mathbb {R} ^{n}\rightarrow \mathbb {R}$ such that $V(f(n))$ is monotonic in $n$ . If the $V$ satisfies the conditions to be a Lyapunov function then $f$ is convergent. Lyapunov's theorem is normally stated for ordinary differential equations, but can also be applied to sequences of iterates by replacing derivatives with discrete differences.

The basic requirements on $L$ are that

$V(f(n+1))-V(f(n))<0$ for $f(n)\neq 0$ and $V(0)=0$ (or ${\dot {V}}(x)<0$ for $x\neq 0$ )
$V(x)>0$ for all $x\neq 0$ and $V(0)=0$
$V$ be "radially unbounded", so that $V(x)$ goes to infinity for any sequence with $||x||$ that tends to infinity.

In many cases, a Lyapunov function of the form $L(x)=x^{T}Ax$ can be found, although more complex forms are also used.

For delay differential equations, a similar approach applies with Lyapunov functions replaced by Lyapunov functionals also called Lyapunov-Krasovskii functionals.

If the inequality in the condition 1 is weak, LaSalle's invariance principle may be used.

Convergence of sequences of functions

To consider the convergence of sequences of functions, it is necessary to define a distance between functions to replace the Euclidean norm. These often include

Convergence in the norm (strong convergence) -- a function norm, such as $\||g||_{f}=\int _{x\in A}||g(x)||dx$ is defined, and convergence occurs if $||f(n)-f_{\infty }||_{f}\rightarrow 0$ . For this case, all of the above techniques can be applied with this function norm.
Pointwise convergence -- convergence occurs if for each $x$ , $f_{n}(x)\rightarrow f_{n}(x_{\infty })$ . For this case, the above techniques can be applied for each point $x$ with the norm appropriate for $f(x)$ .
uniform convergence -- In pointwise convergence, some (open) regions can converge arbitrarily slowly. With uniform convergence, there is a fixed convergence rate such that all points converge at least that fast. Formally, $\lim _{n\to \infty }\,\sup\{\,\left|f_{n}(x)-f_{\infty }(x)\right|:x\in A\,\}=0,$ where $A$ is the domain of each $f_{n}$ .

Convergence of random variables

Random variables are more complicated than simple elements of $\mathbb {R} ^{n}$ . (Formally, a random variable is a mapping $x:\Omega \rightarrow V$ from an event space $\Omega$ to a value space $V$ . The value space may be $\mathbb {R} ^{n}$ , such as the roll of a dice, and such a random variable is often spoken of informally as being in $\mathbb {R} ^{n}$ , but convergence of sequence of random variables corresponds to convergence of the sequence of functions, or the distributions, rather than the sequence of values.)

There are multiple types of convergence, depending on the how the distance between functions is measured.

Convergence in distribution -- pointwise convergence of the distribution functions of the random variables to the limit
Convergence in probability
Almost sure convergence -- pointwise convergence of the mappings $x_{n}:\Omega \rightarrow V$ to the limit, except at a set in $\Omega$ with measure 0 in the limit.
Convergence in the mean

Each has its own proof techniques, which are beyond the current scope of this article.

Topological convergence

For all of the above techniques, some form the basic analytic definition of convergence above applies. However, topology has its own definition of convergence. For example, it a non-hausdorff space, it is possible for a sequence to converge to multiple different limits.

Convergence in Rn