In probability theory , it is possible to approximate the moments of a function f of a random variable X using Taylor expansions , provided that f is sufficiently differentiable and that the moments of X are finite.
Monte Carlo simulations .
Given
μ
X
{\displaystyle \mu _{X}}
σ
X
2
{\displaystyle \sigma _{X}^{2}}
X
{\displaystyle X}
[ 1] expected value of
f
(
X
)
{\displaystyle f(X)}
E
[
f
(
X
)
]
=
E
[
f
(
μ
X
+
(
X
−
μ
X
)
)
]
≈
E
[
f
(
μ
X
)
+
f
′
(
μ
X
)
(
X
−
μ
X
)
+
1
2
f
″
(
μ
X
)
(
X
−
μ
X
)
2
]
=
f
(
μ
X
)
+
f
′
(
μ
X
)
E
[
X
−
μ
X
]
+
1
2
f
″
(
μ
X
)
E
[
(
X
−
μ
X
)
2
]
.
{\displaystyle {\begin{aligned}\operatorname {E} \left[f(X)\right]&{}=\operatorname {E} \left[f\left(\mu _{X}+\left(X-\mu _{X}\right)\right)\right]\\&{}\approx \operatorname {E} \left[f(\mu _{X})+f'(\mu _{X})\left(X-\mu _{X}\right)+{\frac {1}{2}}f''(\mu _{X})\left(X-\mu _{X}\right)^{2}\right]\\&{}=f(\mu _{X})+f'(\mu _{X})\operatorname {E} \left[X-\mu _{X}\right]+{\frac {1}{2}}f''(\mu _{X})\operatorname {E} \left[\left(X-\mu _{X}\right)^{2}\right].\end{aligned}}}
Since
E
[
X
−
μ
X
]
=
0
,
{\displaystyle E[X-\mu _{X}]=0,}
E
[
(
X
−
μ
X
)
2
]
{\displaystyle E[(X-\mu _{X})^{2}]}
σ
X
2
{\displaystyle \sigma _{X}^{2}}
E
[
f
(
X
)
]
≈
f
(
μ
X
)
+
f
″
(
μ
X
)
2
σ
X
2
{\displaystyle \operatorname {E} \left[f(X)\right]\approx f(\mu _{X})+{\frac {f''(\mu _{X})}{2}}\sigma _{X}^{2}}
It is possible to generalize this to functions of more than one variable using multivariate Taylor expansions . For example,[ 2]
E
[
X
Y
]
≈
E
[
X
]
E
[
Y
]
−
cov
[
X
,
Y
]
E
[
Y
]
2
+
E
[
X
]
E
[
Y
]
3
var
[
Y
]
{\displaystyle \operatorname {E} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]}}-{\frac {\operatorname {cov} \left[X,Y\right]}{\operatorname {E} \left[Y\right]^{2}}}+{\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {var} \left[Y\right]}
Similarly,[ 1]
var
[
f
(
X
)
]
≈
(
f
′
(
E
[
X
]
)
)
2
var
[
X
]
=
(
f
′
(
μ
X
)
)
2
σ
X
2
−
1
4
(
f
″
(
μ
X
)
)
2
σ
X
4
{\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}-{\frac {1}{4}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}}
The above is obtained using a second order approximation, following the method used in estimating the first moment. It will be a poor approximation in cases where
f
(
X
)
{\displaystyle f(X)}
delta method .
Indeed, we take
E
[
f
(
X
)
]
≈
f
(
μ
X
)
+
f
″
(
μ
X
)
2
σ
X
2
{\displaystyle \operatorname {E} \left[f(X)\right]\approx f(\mu _{X})+{\frac {f''(\mu _{X})}{2}}\sigma _{X}^{2}}
With
f
(
X
)
=
g
(
X
)
2
{\displaystyle f(X)=g(X)^{2}}
E
[
Y
2
]
{\displaystyle \operatorname {E} \left[Y^{2}\right]}
var
[
Y
]
=
E
[
Y
2
]
−
μ
Y
2
{\displaystyle \operatorname {var} \left[Y\right]=\operatorname {E} \left[Y^{2}\right]-\mu _{Y}^{2}}
An example is,[ 2]
var
[
X
Y
]
≈
var
[
X
]
E
[
Y
]
2
−
2
E
[
X
]
E
[
Y
]
3
cov
[
X
,
Y
]
+
E
[
X
]
2
E
[
Y
]
4
var
[
Y
]
.
{\displaystyle \operatorname {var} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {var} \left[X\right]}{\operatorname {E} \left[Y\right]^{2}}}-{\frac {2\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {cov} \left[X,Y\right]+{\frac {\operatorname {E} \left[X\right]^{2}}{\operatorname {E} \left[Y\right]^{4}}}\operatorname {var} \left[Y\right].}
The second order approximation, when X follows a normal distribution, is:[ 3]
var
[
f
(
X
)
]
≈
(
f
′
(
E
[
X
]
)
)
2
var
[
X
]
+
(
f
″
(
E
[
X
]
)
)
2
2
(
var
[
X
]
)
2
=
(
f
′
(
μ
X
)
)
2
σ
X
2
+
1
2
(
f
″
(
μ
X
)
)
2
σ
X
4
+
(
f
′
(
μ
X
)
)
(
f
‴
(
μ
X
)
)
σ
X
4
{\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]+{\frac {\left(f''(\operatorname {E} \left[X\right])\right)^{2}}{2}}\left(\operatorname {var} \left[X\right]\right)^{2}=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}+{\frac {1}{2}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}+\left(f'(\mu _{X})\right)\left(f'''(\mu _{X})\right)\sigma _{X}^{4}}
First product moment [ edit ] To find a second-order approximation for the covariance of functions of two random variables (with the same function applied to both), one can proceed as follows. First, note that
cov
[
f
(
X
)
,
f
(
Y
)
]
=
E
[
f
(
X
)
f
(
Y
)
]
−
E
[
f
(
X
)
]
E
[
f
(
Y
)
]
{\displaystyle \operatorname {cov} \left[f(X),f(Y)\right]=\operatorname {E} \left[f(X)f(Y)\right]-\operatorname {E} \left[f(X)\right]\operatorname {E} \left[f(Y)\right]}
E
[
f
(
X
)
]
{\displaystyle \operatorname {E} \left[f(X)\right]}
E
[
f
(
X
)
f
(
Y
)
]
{\displaystyle \operatorname {E} \left[f(X)f(Y)\right]}
f
(
X
)
f
(
Y
)
{\displaystyle f(X)f(Y)}
f
(
X
)
f
(
Y
)
≈
f
(
μ
X
)
f
(
μ
Y
)
+
(
X
−
μ
X
)
f
′
(
μ
X
)
f
(
μ
Y
)
+
(
Y
−
μ
Y
)
f
(
μ
X
)
f
′
(
μ
Y
)
+
1
2
[
(
X
−
μ
X
)
2
f
″
(
μ
X
)
f
(
μ
Y
)
+
2
(
X
−
μ
X
)
(
Y
−
μ
Y
)
f
′
(
μ
X
)
f
′
(
μ
Y
)
+
(
Y
−
μ
Y
)
2
f
(
μ
X
)
f
″
(
μ
Y
)
]
{\displaystyle {\begin{aligned}f(X)f(Y)&{}\approx f(\mu _{X})f(\mu _{Y})+(X-\mu _{X})f'(\mu _{X})f(\mu _{Y})+(Y-\mu _{Y})f(\mu _{X})f'(\mu _{Y})+{\frac {1}{2}}\left[(X-\mu _{X})^{2}f''(\mu _{X})f(\mu _{Y})+2(X-\mu _{X})(Y-\mu _{Y})f'(\mu _{X})f'(\mu _{Y})+(Y-\mu _{Y})^{2}f(\mu _{X})f''(\mu _{Y})\right]\end{aligned}}}
Taking expectation of the above and simplifying—making use of the identities
E
(
X
2
)
=
var
(
X
)
+
[
E
(
X
)
]
2
{\displaystyle \operatorname {E} (X^{2})=\operatorname {var} (X)+\left[\operatorname {E} (X)\right]^{2}}
E
(
X
Y
)
=
cov
(
X
,
Y
)
+
[
E
(
X
)
]
[
E
(
Y
)
]
{\displaystyle \operatorname {E} (XY)=\operatorname {cov} (X,Y)+\left[\operatorname {E} (X)\right]\left[\operatorname {E} (Y)\right]}
E
[
f
(
X
)
f
(
Y
)
]
≈
f
(
μ
X
)
f
(
μ
Y
)
+
f
′
(
μ
X
)
f
′
(
μ
Y
)
cov
(
X
,
Y
)
+
1
2
f
″
(
μ
X
)
f
(
μ
Y
)
var
(
X
)
+
1
2
f
(
μ
X
)
f
″
(
μ
Y
)
var
(
Y
)
{\displaystyle \operatorname {E} \left[f(X)f(Y)\right]\approx f(\mu _{X})f(\mu _{Y})+f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)+{\frac {1}{2}}f''(\mu _{X})f(\mu _{Y})\operatorname {var} (X)+{\frac {1}{2}}f(\mu _{X})f''(\mu _{Y})\operatorname {var} (Y)}
cov
[
f
(
X
)
,
f
(
Y
)
]
≈
f
(
μ
X
)
f
(
μ
Y
)
+
f
′
(
μ
X
)
f
′
(
μ
Y
)
cov
(
X
,
Y
)
+
1
2
f
″
(
μ
X
)
f
(
μ
Y
)
var
(
X
)
+
1
2
f
(
μ
X
)
f
″
(
μ
Y
)
var
(
Y
)
−
[
f
(
μ
X
)
+
1
2
f
″
(
μ
X
)
var
(
X
)
]
[
f
(
μ
Y
)
+
1
2
f
″
(
μ
Y
)
var
(
Y
)
]
=
f
′
(
μ
X
)
f
′
(
μ
Y
)
cov
(
X
,
Y
)
−
1
4
f
″
(
μ
X
)
f
″
(
μ
Y
)
var
(
X
)
var
(
Y
)
{\displaystyle {\begin{aligned}\operatorname {cov} \left[f(X),f(Y)\right]&{}\approx f(\mu _{X})f(\mu _{Y})+f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)+{\frac {1}{2}}f''(\mu _{X})f(\mu _{Y})\operatorname {var} (X)+{\frac {1}{2}}f(\mu _{X})f''(\mu _{Y})\operatorname {var} (Y)-\left[f(\mu _{X})+{\frac {1}{2}}f''(\mu _{X})\operatorname {var} (X)\right]\left[f(\mu _{Y})+{\frac {1}{2}}f''(\mu _{Y})\operatorname {var} (Y)\right]\\&{}=f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)-{\frac {1}{4}}f''(\mu _{X})f''(\mu _{Y})\operatorname {var} (X)\operatorname {var} (Y)\end{aligned}}}
If X is a random vector, the approximations for the mean and variance of
f
(
X
)
{\displaystyle f(X)}
[ 4]
E
(
f
(
X
)
)
=
f
(
μ
X
)
+
1
2
trace
(
H
f
(
μ
X
)
Σ
X
)
var
(
f
(
X
)
)
=
∇
f
(
μ
X
)
t
Σ
X
∇
f
(
μ
X
)
+
1
2
trace
(
H
f
(
μ
X
)
Σ
X
H
f
(
μ
X
)
Σ
X
)
.
{\displaystyle {\begin{aligned}\operatorname {E} (f(X))&=f(\mu _{X})+{\frac {1}{2}}\operatorname {trace} (H_{f}(\mu _{X})\Sigma _{X})\\\operatorname {var} (f(X))&=\nabla f(\mu _{X})^{t}\Sigma _{X}\nabla f(\mu _{X})+{\frac {1}{2}}\operatorname {trace} \left(H_{f}(\mu _{X})\Sigma _{X}H_{f}(\mu _{X})\Sigma _{X}\right).\end{aligned}}}
Here
∇
f
{\displaystyle \nabla f}
H
f
{\displaystyle H_{f}}
gradient and the Hessian matrix respectively, and
Σ
X
{\displaystyle \Sigma _{X}}
covariance matrix of X .
^ a b Haym Benaroya, Seon Mi Han, and Mark Nagurka. Probability Models in Engineering and Science . CRC Press, 2005, p166.
^ a b van Kempen, G.m.p.; van Vliet, L.j. (1 April 2000). "Mean and Variance of Ratio Estimators Used in Fluorescence Ratio Imaging" . Cytometry . 39 (4): 300–305. doi :10.1002/(SICI)1097-0320(20000401)39:4<300::AID-CYTO8>3.0.CO;2-O . Retrieved 2024-08-14 . ^ Hendeby, Gustaf; Gustafsson, Fredrik. "ON NONLINEAR TRANSFORMATIONS OF GAUSSIAN DISTRIBUTIONS" (PDF) . Retrieved 5 October 2017 . ^ Rego, Bruno V.; Weiss, Dar; Bersi, Matthew R.; Humphrey, Jay D. (14 December 2021). "Uncertainty quantification in subject‐specific estimation of local vessel mechanical properties" . International Journal for Numerical Methods in Biomedical Engineering 37 (12): e3535. doi :10.1002/cnm.3535 . ISSN 2040-7939 . PMC 9019846 PMID 34605615 .
![{\displaystyle {\begin{aligned}\operatorname {E} \left[f(X)\right]&{}=\operatorname {E} \left[f\left(\mu _{X}+\left(X-\mu _{X}\right)\right)\right]\\&{}\approx \operatorname {E} \left[f(\mu _{X})+f'(\mu _{X})\left(X-\mu _{X}\right)+{\frac {1}{2}}f''(\mu _{X})\left(X-\mu _{X}\right)^{2}\right]\\&{}=f(\mu _{X})+f'(\mu _{X})\operatorname {E} \left[X-\mu _{X}\right]+{\frac {1}{2}}f''(\mu _{X})\operatorname {E} \left[\left(X-\mu _{X}\right)^{2}\right].\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/d9453f100161227140288817a070c40ad87c22b2)
![{\displaystyle E[X-\mu _{X}]=0,}](https://wikimedia.org/api/rest_v1/media/math/render/svg/000acb6340500505b2c39bf53b182169aa6b85d7)
![{\displaystyle E[(X-\mu _{X})^{2}]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/4d1906e7bfddbbb4fa3b71cf84887c95cb180995)
![{\displaystyle \operatorname {E} \left[f(X)\right]\approx f(\mu _{X})+{\frac {f''(\mu _{X})}{2}}\sigma _{X}^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/27108a7bd1d981b02fa89424930f479ec6fe8d4e)
![{\displaystyle \operatorname {E} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]}}-{\frac {\operatorname {cov} \left[X,Y\right]}{\operatorname {E} \left[Y\right]^{2}}}+{\frac {\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {var} \left[Y\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/bf8b82f9c15c42b3fa41c397dd3b6a1d67735539)
![{\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}-{\frac {1}{4}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/94fb3c55d082462f42a9a9b9cb7bae4d1e7c3053)
![{\displaystyle \operatorname {E} \left[Y^{2}\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/903a9bbb4f14aa8ecd2db07dd1180baa01d6b30f)
![{\displaystyle \operatorname {var} \left[Y\right]=\operatorname {E} \left[Y^{2}\right]-\mu _{Y}^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/93ec1a80c2e68b3f7b91aaffd5c0655600c10a4f)
![{\displaystyle \operatorname {var} \left[{\frac {X}{Y}}\right]\approx {\frac {\operatorname {var} \left[X\right]}{\operatorname {E} \left[Y\right]^{2}}}-{\frac {2\operatorname {E} \left[X\right]}{\operatorname {E} \left[Y\right]^{3}}}\operatorname {cov} \left[X,Y\right]+{\frac {\operatorname {E} \left[X\right]^{2}}{\operatorname {E} \left[Y\right]^{4}}}\operatorname {var} \left[Y\right].}](https://wikimedia.org/api/rest_v1/media/math/render/svg/011aff1036d96609635f44161e05afa36d783d19)
![{\displaystyle \operatorname {var} \left[f(X)\right]\approx \left(f'(\operatorname {E} \left[X\right])\right)^{2}\operatorname {var} \left[X\right]+{\frac {\left(f''(\operatorname {E} \left[X\right])\right)^{2}}{2}}\left(\operatorname {var} \left[X\right]\right)^{2}=\left(f'(\mu _{X})\right)^{2}\sigma _{X}^{2}+{\frac {1}{2}}\left(f''(\mu _{X})\right)^{2}\sigma _{X}^{4}+\left(f'(\mu _{X})\right)\left(f'''(\mu _{X})\right)\sigma _{X}^{4}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/14c6f8695bf69e9302602a388257f566ba6f1891)
![{\displaystyle \operatorname {cov} \left[f(X),f(Y)\right]=\operatorname {E} \left[f(X)f(Y)\right]-\operatorname {E} \left[f(X)\right]\operatorname {E} \left[f(Y)\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/eb30d1f4b08217ad7dc8430b0af14b93f0a27a02)
![{\displaystyle \operatorname {E} \left[f(X)\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/904f0b10ffff22db15df5d7556258e70aeaa9693)
![{\displaystyle \operatorname {E} \left[f(X)f(Y)\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/306d2c5c194290007ebe5a25f48e47107ae5a61d)
![{\displaystyle {\begin{aligned}f(X)f(Y)&{}\approx f(\mu _{X})f(\mu _{Y})+(X-\mu _{X})f'(\mu _{X})f(\mu _{Y})+(Y-\mu _{Y})f(\mu _{X})f'(\mu _{Y})+{\frac {1}{2}}\left[(X-\mu _{X})^{2}f''(\mu _{X})f(\mu _{Y})+2(X-\mu _{X})(Y-\mu _{Y})f'(\mu _{X})f'(\mu _{Y})+(Y-\mu _{Y})^{2}f(\mu _{X})f''(\mu _{Y})\right]\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f1967664e3eb99df8bca7a26321484a0cbdeb07f)
![{\displaystyle \operatorname {E} (X^{2})=\operatorname {var} (X)+\left[\operatorname {E} (X)\right]^{2}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/787ab52bc02aee12a354b05e722f54c59e78e48e)
![{\displaystyle \operatorname {E} (XY)=\operatorname {cov} (X,Y)+\left[\operatorname {E} (X)\right]\left[\operatorname {E} (Y)\right]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/3dcf130a425a5c5c545cdecd7ecbda55eaff943d)
![{\displaystyle \operatorname {E} \left[f(X)f(Y)\right]\approx f(\mu _{X})f(\mu _{Y})+f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)+{\frac {1}{2}}f''(\mu _{X})f(\mu _{Y})\operatorname {var} (X)+{\frac {1}{2}}f(\mu _{X})f''(\mu _{Y})\operatorname {var} (Y)}](https://wikimedia.org/api/rest_v1/media/math/render/svg/902dd8ed2792705004ada3a57b5fe1fa288b8b76)
![{\displaystyle {\begin{aligned}\operatorname {cov} \left[f(X),f(Y)\right]&{}\approx f(\mu _{X})f(\mu _{Y})+f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)+{\frac {1}{2}}f''(\mu _{X})f(\mu _{Y})\operatorname {var} (X)+{\frac {1}{2}}f(\mu _{X})f''(\mu _{Y})\operatorname {var} (Y)-\left[f(\mu _{X})+{\frac {1}{2}}f''(\mu _{X})\operatorname {var} (X)\right]\left[f(\mu _{Y})+{\frac {1}{2}}f''(\mu _{Y})\operatorname {var} (Y)\right]\\&{}=f'(\mu _{X})f'(\mu _{Y})\operatorname {cov} (X,Y)-{\frac {1}{4}}f''(\mu _{X})f''(\mu _{Y})\operatorname {var} (X)\operatorname {var} (Y)\end{aligned}}}](https://wikimedia.org/api/rest_v1/media/math/render/svg/43a68317901c1d4652df64c01890265e1d49b2d4)
