Wikipedia:Reference desk/Archives/Mathematics/2018 January 31

Welcome to the Wikipedia Mathematics Reference Desk Archives
The page you are currently viewing is a transcluded archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

January 31

Expectation value and standard deviation of the largest of n draws from standard normal distribution

I only found closed form expressions for n = 2 and n = 3. Are there closed form expression for the expectation value and standard deviation for larger n or perhaps even closed form expressions for general n?

For n = 2, I find that the expectation value is ${\frac {1}{\sqrt {\pi }}}$ and the standard deviation is ${\sqrt {\frac {\pi -1}{\pi }}}$.

For n = 3, the expectation value is ${\frac {3}{2{\sqrt {\pi }}}}$, but I don't have a closed form for the standard deviation.

Count Iblis (talk) 03:35, 31 January 2018 (UTC)

Just a bit of clarification, when you say you found the values for n=2 and n=3, did you find the values in a book or online (if so where?) or were you able to derive them? Also, by standard normal I'm assuming you mean the distribution with density

$\varphi (x)={\frac {1}{\sqrt {2\pi }}}e^{-{\frac {1}{2}}x^{2}}$

and not the 'other' standard

$\varphi (x)={\frac {e^{-x^{2}}}{\sqrt {\pi }}}.$

--RDBury (talk) 08:35, 31 January 2018 (UTC)

I was able to derive the closed form expressions for the expected values for n=2 and n=3, and reduce the value for higher n to a single integral, but I wasn't able to get this into closed form for n=4. Using the same method I was able to get closed form expressions for the 2nd moment for n=2 and n=3 (new), but again it doesn't seem to simplify for n=4. I'll adopt the notation used in Normal distribution, namely

$\varphi (x)={\frac {1}{\sqrt {2\pi }}}e^{-{\frac {1}{2}}x^{2}}$

and

$\Phi (x)=\int _{-\infty }^{x}\varphi (t)\,{\rm {d}}t.$

The CDF for the maximum of n values is $\Phi ^{n}(x).$ and so the corresponding density function is

${\frac {\rm {d}}{{\rm {d}}x}}\Phi ^{n}(x)=n\Phi ^{n-1}(x)\varphi (x).$

We're interested in the kth moments for these distributions:

$M_{n,k}=\int _{-\infty }^{+\infty }x^{k}n\Phi ^{n-1}(x)\varphi (x)\,{\rm {d}}x,$

specifically for k=1 for the expected value and for k=2 to get the standard deviation. It's clear that $M_{n,0}=1.$ Also, using ${\frac {\rm {d}}{{\rm {d}}x}}\varphi (x)=-x\varphi (x)$ and integrating by parts

$M_{1,k}=\int _{-\infty }^{+\infty }x^{k}\varphi (x)\,{\rm {d}}x=-\int _{-\infty }^{+\infty }x^{k-1}\varphi '(x)\,{\rm {d}}x=(k-1)\int _{-\infty }^{+\infty }x^{k-2}\varphi (x)\,{\rm {d}}x=(k-1)M_{1,k-2}$

from which

$M_{1,k}={\begin{cases}0&{\text{if }}k{\text{ is odd,}}\\(k-1)!!&{\text{if }}k{\text{ is even.}}\end{cases}}$

(This is well known but it's good to have a bit of a warm-up.) For n>1 and k=1

$M_{n,1}=\int _{-\infty }^{+\infty }xn\Phi ^{n-1}(x)\varphi (x)\,{\rm {d}}x=-n\int _{-\infty }^{+\infty }\Phi ^{n-1}(x)\varphi '(x)\,{\rm {d}}x$

and integrating by parts gives

$M_{n,1}=n(n-1)\int _{-\infty }^{+\infty }\Phi ^{n-2}(x)\varphi ^{2}(x)\,{\rm {d}}x.$

For n=2 this is

$M_{2,1}=2\int _{-\infty }^{+\infty }\varphi ^{2}(x)\,{\rm {d}}x={\frac {2}{\sqrt {2\pi }}}\int _{-\infty }^{+\infty }\varphi ({\sqrt {2}}x)\,{\rm {d}}x={\frac {1}{\sqrt {\pi }}}$

as given above.

For n=3 this is

$M_{3,1}=6\int _{-\infty }^{+\infty }\Phi (x)\varphi ^{2}(x)\,{\rm {d}}x=6\int _{-\infty }^{+\infty }(\Phi (x)-{\frac {1}{2}}+{\frac {1}{2}})\varphi ^{2}(x)\,{\rm {d}}x=6\int _{-\infty }^{+\infty }{\frac {1}{2}}\varphi ^{2}(x)\,{\rm {d}}x$

since $(\Phi (x)-{\frac {1}{2}})\varphi ^{2}(x)$ is odd. Then

$M_{3,1}=3\int _{-\infty }^{+\infty }\varphi ^{2}(x)\,{\rm {d}}x={\frac {3}{2{\sqrt {\pi }}}}$

by the previous result.

For n=4 the integral is

$M_{4,1}=12\int _{-\infty }^{+\infty }\Phi ^{2}(x)\varphi ^{2}(x)\,{\rm {d}}x$

which may simplify using some of the formulas in List of integrals of Gaussian functions, but I don't see the way forward atm. It appears the integrals get more and more difficult as n increases, so I'm guessing that while there may be some recursive formula, a closed form expression is unlikely.

For k=2 the integral is

$M_{n,2}=n\int _{-\infty }^{+\infty }x^{2}\Phi ^{n-1}(x)\varphi (x)\,{\rm {d}}x,$

and using integration by parts as before this becomes

$M_{n,2}=n\int _{-\infty }^{+\infty }\Phi ^{n-1}(x)\varphi (x)\,{\rm {d}}x+n(n-1)\int _{-\infty }^{+\infty }x\Phi ^{n-2}(x)\varphi ^{2}(x)\,{\rm {d}}x=1+n(n-1)\int _{-\infty }^{+\infty }x\Phi ^{n-2}(x)\varphi ^{2}(x)\,{\rm {d}}x.$

For n=2 the integral

$\int _{-\infty }^{+\infty }x\varphi ^{2}(x)\,{\rm {d}}x.$

is 0 because the integrand is odd, therefor M(2, 2) = 1 agreeing with the value for the standard deviation given above.

For n=3 set

$I=\int _{-\infty }^{+\infty }x\Phi (x)\varphi ^{2}(x)\,{\rm {d}}x.$

Then

$I=-\int _{-\infty }^{+\infty }\Phi (x)\varphi (x)\varphi '(x)\,{\rm {d}}x$

and integrating by parts gives

$I=\int _{-\infty }^{+\infty }\varphi ^{3}(x),{\rm {d}}x+\int _{-\infty }^{+\infty }\Phi (x)\varphi (x)\varphi '(x)\,{\rm {d}}x=\int _{-\infty }^{+\infty }\varphi ^{3}(x),{\rm {d}}x-I,$

so

$2I=\int _{-\infty }^{+\infty }\varphi ^{3}(x),{\rm {d}}x={\frac {1}{2\pi }}\int _{-\infty }^{+\infty }\varphi ({\sqrt {3}}x){\rm {d}}x={\frac {1}{2{\sqrt {3}}\pi }}.$

Therefore $I={\frac {1}{4{\sqrt {3}}\pi }}$ and $M_{3,2}=1+{\frac {\sqrt {3}}{2\pi }},$ from which the standard deviation for the n=3 can be computed. (In this case I don't have an answer to compare to so take this value modulo errors in algebra. Maybe someone can check the work or do a quick computer simulation to verify.) Sorry if the post is over-long, but perhaps a bit of verbosity is acceptable since the forum has been a bit slow in the last week. --RDBury (talk) 14:06, 31 January 2018 (UTC)

You wrote "The CDF for the maximum of n values is $\Phi ^{n}(x).$". Can you please explain why that's true or link an article or external resource which does? (I fixed your "[Normal distribution]" link and linked CDF. Hope that's OK.) -- ToE 17:50, 31 January 2018 (UTC)

That cannot be true, if only because that function is symmetric with respect to 0 (which it should not be - the expected value of a maximum of independent nontrivial variables is certainly superior to the expected value of any of the variables). I suspect an abuse of notation slipped into an incorrect formula. Tigraan^{Click here to contact me} 18:16, 31 January 2018 (UTC)

What? That's the cumulative distribution function, not the probability density function or error function. It and its powers (even n=2) won't be symmetric with respect to 0. The powers will shift a given cumulative probability farther right. It behaves as I'd expect the CDF for a maximum of n values would. I just don't see how to prove it. -- ToE 18:43, 31 January 2018 (UTC)

Huh, yes, sorry. Still, it would be good to know whether it is true. Tigraan^{Click here to contact me} 12:19, 1 February 2018 (UTC)

I actually gave this problem as an exercise for my students this semester (analytically for a uniform distribution, numerically for the normal distribution). We derived/justified the pdf as follows: x is the maximum sample value if (i) x is a sample value, this gives a factor $\varphi (x)$ with a combinatorial factor N because it doesn't matter where in the sample it appears; (ii) n-1 sample values must be $\leq x$, this gives a factor $\Phi (x)^{n-1}$. This agrees with the derivative of the cdf given above. Our numerical results (for the expectation only) agree with those of Count Iblis, and it's nice to see to what extent the problem can be tackled analytically. --Wrongfilter (talk) 12:38, 1 February 2018 (UTC)

• I tackled the problem from a different angle, with the trick to use weird integration domains instead of weird functions to integrate. The result involves an integral with powers of the error function; I am not sure whether a closed-form formula or even a recurrence relation exists, but others may weight in.

Computations collapsed for readability - TigraanClick here to contact me 18:12, 31 January 2018 (UTC)

Let $X_{1},X_{2},...X_{n}$ be independent random variables with p.d.f. $\phi (t)$. To evaluate the moment of order k (in our case, we want k=0,1), we would usually integrate over $\mathbb {R} ^{n}$ (for all possible values of the $X_{i}$) the function $\phi (X_{1})\phi (X_{2})...\phi (X_{n})max(X_{1},...X_{n})^{k}$. The key insight is that instead, we can integrate the simpler function $\phi (X_{1})\phi (X_{2})...\phi (X_{n})X_{1}^{k}$ over the domain where X1 is the maximum, and divide by the weight of that domain (i.e. the integral of $\phi (X_{1})\phi (X_{2})...\phi (X_{n})$) (that weight is 1 in the previous case, assuming the p.d.f. are normalized). Hence we get: $M_{n,k}=\int _{X_{1}\in =]-\infty ,+\infty [}\int _{X_{2},...X_{n}\in =]-\infty ,X_{1}]}\phi (X_{1})\phi (X_{2})...\phi (X_{n})X_{1}^{k}dX_{1}dX_{2}...dX_{n}/D_{n}$ where the domain size is $D_{n}=\int _{X_{1}\in =]-\infty ,+\infty [}\int _{X_{2},...X_{n}\in =]-\infty ,X_{1}]}\phi (X_{1})\phi (X_{2})...\phi (X_{n})dX_{1}dX_{2}...dX_{n}$ Notice that the integration over all variables is independent (except for the bounds of i≥2 that depend on X1); moreover, all the Xi,i≥2 have the same p.d.f. so this simplifies drastically. Using the same notation as RDBury above with $\Phi (t)=\int _{-\infty }^{t}\phi (t)dt$ $D_{n}*M_{n,k}=\int _{X_{1}\in =]-\infty ,+\infty [}X_{1}^{k}dX_{1}\phi (X_{1})\left(\int _{X\in =]-\infty ,X_{1}]}\phi (X)dX\right)^{n-1}=\int _{-\infty }^{+\infty }X^{k}\Phi (X)^{n-1}\phi (X)dX$ and similarly $D_{n}=\int _{-\infty }^{+\infty }\Phi (X)^{n-1}\phi (X)dX$ It all comes down to the p.d.f. in use. If it was uniform distribution over an interval, that is a simple polynomial integral, but the current case calls for a gaussian distribution, so $\Phi (X)$ is the error function (maybe with a scaling factor).

Final result: with RDBury's notations, $M_{n,k}=\int _{-\infty }^{+\infty }X^{k}\Phi (X)^{n-1}\phi (X)dX/\int _{-\infty }^{+\infty }\Phi (X)^{n-1}\phi (X)dX$. I am not sure how much further algebra-lifting and List of integrals of Gaussian functions can get you, but that is good enough that any computer should have no trouble to give you a 10-digit approximation for reasonable n/k. Tigraan^{Click here to contact me} 18:12, 31 January 2018 (UTC)

What an idiot... The integrand in the denominator has obviously the primitive ${\frac {1}{n}}\Phi ^{n}$, so it's easy to integrate:

$D_{n}=\int _{-\infty }^{+\infty }\Phi (X)^{n-1}\phi (X)dX=\left[{\frac {1}{n}}\Phi (X)^{n}\right]_{X=-\infty }^{X=+\infty }={\frac {1}{n}}$

So the result simplifies to $M_{n,k}=n\int _{-\infty }^{+\infty }X^{k}\Phi (X)^{n-1}\phi (X)dX$. It looks like a good candidate for integration by parts, but for k≥1, the parts are infinite (the term $k\int _{-\infty }^{+\infty }X^{k-1}\Phi (X)^{n}dX$ will not converge, since the integrand does not approach 0 for large X). Tigraan^{Click here to contact me} 13:07, 1 February 2018 (UTC)

First, for a possibly simpler explanation of the cdf of the max of independent variables, this holds for any distribution and even holds if the variables have different distribution. Say variable X as cdf U and Y has cdf V. The the cdf for max(X, Y) is given by W(x) = P(max(X,Y)≤x) = P(X≤x and Y≤x) = P(X≤x)⋅P(Y≤x) (since the variables are independent) = U(x)⋅V(x). You can use induction to prove the corresponding statement for any finite number of variables.

Also, to follow up on some unfinished items above. I have since worked up a very basic Python program to estimate the values; basically it just finds the average of X, X², X³ and X⁴ for 1,000,000 values of X, where X is the maximum of 2, 3 or 4 random values chosen with the standard normal distribution. This only produces about 2 or 3 digits of accuracy but that's good enough for a sanity check. The results match the value of M_3,2 given above so I'm much more confident that it's the correct value.

I was also able to find the exact value of M_4,1 as $6\tan ^{-1}{\sqrt {2}}\cdot \pi ^{-{\tfrac {3}{2}}}$ or about 1.029. I agree with Tigraan that to go much further it would be easier to use numerical methods, or at least use a computer algebra package to do the calculus. I've looked at Alpha for this but it seems they use the 'other' standard for the standard distribution and converting the Φ(x) I'm using to their erf(x) would be an issue. I\m not enough of a Mathematica user to know if there's a way around this. --RDBury (talk) 15:22, 1 February 2018 (UTC)

Thanks for looking into this! It looks less hopeless to try to find more exact expressions for the moments than when I first looked into this. Count Iblis (talk) 21:58, 4 February 2018 (UTC)

Negative and fractional exponents in cardinal exponentiation

What are the conditions for the existence of negative and fractional exponents in cardinal exponentiation? The info re this aspect from that article (section) is not very detailed! (Thanks!)--5.2.200.163 (talk) 15:30, 31 January 2018 (UTC)

The conditions are simple: You can't do it at all. Negative numbers and fractions are not cardinal numbers. --Trovatore (talk) 06:28, 1 February 2018 (UTC)

Then what is said there in article about roots and logarithms? When roots are involved, doesn't that automatically involve fractional exponents?

What exactly is the relation between cartesian products and cardinal exponents? Are negative and fractional exponents also excluded from cartesian products?--5.2.200.163 (talk) 15:55, 1 February 2018 (UTC)

No. Roots are the answer to the question "Given b and c, what is the value of a such that $a^{b}=c$?". Whenever there is a concept of exponentiation, it makes sense to talk about roots (even if they don't necessarily exist).

For positive real numbers, it so happens that fractions are a thing and the bth root of c is $c^{1/b}$. For cardinal numbers, there are no fractional exponents, but it doesn't mean there is no concept of roots.

The relation between finite cartesian products and cardinal numbers is simple enough - for any set X, we have $|X^{n}|=|X|^{n}$. For infinitary products this might get more complicated so I'd best leave that to Trovatore. In any case, I don't know of any notion of fractional or negative cartesian exponents. -- Meni Rosenfeld (talk) 19:15, 1 February 2018 (UTC)

The "Roots" subsection makes a simple and accurate assertion:

Assuming the axiom of choice and, given an infinite cardinal κ and a finite cardinal μ greater than 0, the cardinal ν satisfying $\nu ^{\mu }=\kappa$ will be κ.

That claim is true, and I suppose if you like, you can phrase it as "the μ^th root of κ is κ". However, I am not aware that anyone working in the field actually does phrase it that way, in practice.

The "logarithms" section, on the other hand, specifically defines "logarithm" as "the least cardinal number μ such that κ ≤ 2^μ". Whether this definition is standard I cannot tell you. There are three sources cited but I do not have easy access to any of them. All I can tell you is that I do not ever recall seeing this notion referred to as "logarithm" in any serious set-theoretic work, so if you're planning to write a paper that references it you'd better say what you mean, rather than counting on the reader to guess.

In any case, standard terminology or not, this notion of "logarithm" does not require fractional exponents. The corresponding notion on ordinary numbers would not be "log base 2" but rather "ceiling of log base 2". --Trovatore (talk) 19:48, 1 February 2018 (UTC)