Talk:Rao–Blackwell theorem

Lehmann-Scheffé minimum variance

In the article states that if the NEW estimator is complete and sufficient then it is the minimum variance. But doesn't the Lehmann-Scheffé deal specifically with using a complete and sufficent statistic to find a new estimator given an unbiased estimator? ZioX 22:51, 21 March 2007 (UTC)

Looks as if it ought to say if the statistic on which you condition is complete and sufficient, and the estimator you start with is unbiased, then the Rao-Blackwell estimator is the best unbiased estimator. Michael Hardy 22:37, 21 March 2007 (UTC)

Yes, that's what I figured. I didn't want to change without saying anything. ZioX 22:51, 21 March 2007 (UTC)

Changed it. ZioX 21:05, 22 March 2007 (UTC)

Example

Calculating delta_1 is not as trivial as it's being made out to be. At least not to the casual reader. Perhaps something should be said about X_1|sum(X_i) ~ Bin(sum(X_i),1/n)? ZioX 22:56, 21 March 2007 (UTC)

I think correct delta_1 is (1-Sn/n) instead of written (1-1/n)^Sn. Will anyone confirm it? K.Y. 27 November 2008 —Preceding unsigned comment added by 119.240.60.195 (talk) 16:29, 26 November 2008 (UTC)

You're mistaken; see details below. Michael Hardy (talk) 17:07, 26 November 2008 (UTC)

Example

Phone calls arrive at a switchboard according to a Poisson process at an average rate of λ per minute. This rate is not observable, but the numbers X₁, ..., X_n of phone calls that arrived during n successive one-minute periods are observed. It is desired to estimate the probability e^−λ that the next one-minute period passes with no phone calls.

An extremely crude estimator of the desired probability is

$\delta _{0}=\left\{{\begin{matrix}1&{\mbox{if}}\ X_{1}=0\\0&{\mbox{otherwise}}\end{matrix}}\right\},$

i.e., this estimates this probability to be 1 if no phone calls arrived in the first minute and zero otherwise. Despite the apparent limitations of this estimator, the result given by its Rao–Blackwellization is a very good estimator.

The sum

$S_{n}=\sum _{i=1}^{n}X_{i}=X_{1}+\cdots +X_{n}\,\!$

can be readily shown to be a sufficient statistic for λ, i.e., the conditional distribution of the data X₁, ..., X_n, given this sum, does not depend on λ. Therefore, we find the Rao–Blackwell estimator

$\delta _{1}=\operatorname {E} (\delta _{0}|S_{n}).\,\!$

After doing some algebra we have

$\delta _{1}=\left(1-{1 \over n}\right)^{S_{n}}.\,\!$

Algebraic details

We want to evaluate the conditional expected value

$\delta _{1}=\operatorname {E} (\delta _{0}|S_{n}).\,$

So let us consider

${\begin{aligned}\delta _{1}&=\operatorname {E} (\delta _{0}|S_{n}=s)=\Pr(X_{1}=0\mid S_{n}=s)\\\\&={\frac {\Pr(X_{1}=0{\text{ and }}X_{1}+\cdots +X_{n}=s)}{\Pr(S_{n}=s)}}\\\\&={\frac {\Pr(X_{1}=0{\text{ and }}X_{2}+\cdots +X_{n}=s)}{\Pr(S_{n}=s)}}\\\\&={\frac {\Pr(X_{1}=0)\Pr(X_{2}+\cdots +X_{n}=s)}{\Pr(S_{n}=s)}}\\\\&={\frac {\Pr(\operatorname {Poisson} (\lambda )=0)\cdot \Pr(\operatorname {Poisson} ((n-1)\lambda )=s)}{\Pr(\operatorname {Poisson} (n\lambda )=s)}}\\\\&={\frac {e^{-\lambda }\cdot [((n-1)\lambda )^{s}e^{-(n-1)\lambda }/s!]}{(n\lambda )^{s}e^{-n\lambda }/s!}}\\\\&=\left({\frac {n-1}{n}}\right)^{s}=\left(1-{\frac {1}{n}}\right)^{s}.\end{aligned}}$

And so,

$\operatorname {E} (\delta _{1}\mid S_{n})=\left(1-{\frac {1}{n}}\right)^{S_{n}}.$

Michael Hardy (talk) 17:03, 26 November 2008 (UTC)

Thank you very much for your quick response. And the detail is quite clear.

I tried to get this conditional expectation with an binary process on pi:=P(X_1=0|pi)=exp(-lambda). Now I wondering why I got (1-Sn/n) by deleting parameter with

Combin(n-1, s)*pi^(n-1-s)*(1-pi)^s * pi

---

.

Combin(n-1, s)*pi^(n-1-s)*(1-pi)^s * pi + Combin(n-1, s-1)*pi^(n-1-(s-1))*(1-pi)^(s-1) * (1-pi)

Is this substituting (Poisson to Binary) harmful? Now I don't have to adhere it, but I want to confirm.

If you have any comments, please tell me. Thank you in advance.

K.Y. to Michael Hardy —Preceding unsigned comment added by 119.240.60.195 (talk) 19:35, 26 November 2008 (UTC)

mistakes

Sufficiency:

The article states that "a sufficient statistic T(X) for a parameter k is a statistic such that the conditional distribution of the data X, given T(X), depends only on parameter k and not on any other parameters". However, it is the other way around: The conditional distribution does NOT depend on the parameter since T explains it "sufficiently".

Completeness/ idempotence:

T is complete and sufficient does not imply idempotence of the Rao-Blackwell-precedure, as stated in the article.

For example, consider Gaussian random variables X_1,...,X_n with unknown expectation \mu and known variance. Then, it is widely known that the mean of the random variables is sufficient and complete. However, if we consider the median of X_1,...,X_n as an estimator of the expectation, the corresponding Rao-Blackwell-estimator will simply be the mean. Thus, the precedure is not idempotent.

The only way, it stays the same is that the original estimator is measurable with respect to the sigma-algebra induced by the sufficient statistic. —Preceding unsigned comment added by 82.113.106.4 (talk) 00:13, 20 December 2010 (UTC)

References

This article has a message saying that it needs citations. I found some sources, but I'm unsure how reliable they are. One is the transcript of a lecture at University of California, San Diego, and the other is a professor's notes for students from Georgia Institute of Technology. Which, if any, should I include? Weux082690 (talk) 00:28, 29 April 2013 (UTC)