Talk:Confidence interval/Archive 4

Archive 1

Archive 2

Archive 3

Archive 4

Gaussian mislead - other example required

The example given in this page assumes data comes from a gaussian distribution. This is the classical approach to statistics,but is often misleading. In my experience (of analyzing computer simulation outputs), it is not often true.

If I have a set of n independent measurements that come from a distribution F, it is much safer to obtain a confidence interval for the median than for the mean. The confidence interval for the median does not depend on any distributional assumption, and is equally qualified as the mean to give a confidence interval.

Why should this page insist on leading newcomers to compute confidence intervals to the mean when this is unsafe, and there is a better alternative ?

Leboudec (talk) 15:45, 16 July 2012 (UTC)

Confidence limits

Confidence limits redirects to this article, however, the term is not mentioned even once in the article. It would be nice to clarify the relationship between confidence intervals and confidence limits, or, if (as I assume) the limits are the endpoints of the interval, to state this explicitly in the article. Vegard (talk) 09:34, 25 September 2012 (UTC)

Inconsistency in interpretation?

In the “Practical example” section, just above the “Theoretical example” sections starts, we read these two contradictory statements.

First we read this:

One cannot say: "with probability (1 − α) the parameter μ lies in the confidence interval."

Then after a short amount of explanation, we find this:

That is why one can say: "with confidence level 100(1 − α)%, μ lies in the confidence interval."

To me, these two statements are effectively saying the same thing. Any comments? — Preceding unsigned comment added by 203.29.107.155 (talk) 06:25, 26 July 2012 (UTC)

The statements are different, because confidence is not probability. When one says of a procedure to take sample data from a population and construct an interval estimate of a population parameter that there is a 95% confidence of the interval covering the parameter, one is making a statement about the procedure, not about the calculated interval. It means that if this procedure were repeated and lots of interval estimates calculated from lots of sample sets, then 95% of the calculated intervals would be expected to cover the population parameter. It does not mean that one can say of a particular sample set and its calculated interval that this one has a 95% chance of covering the parameter. Still less does it mean that there is a 95% probability of the parameter being within the interval, because in frequentist statistics one can only assign probability values to random variables: the parameter is either within the interval or it isn't - it is not a variable. To ask, what is the probability that the parameter is inside the interval? is a question that does make sense in Bayesian statistics, but that is a different concern: confidence intervals are fundamentally a frequentist concept. Confusing confidence with probability is very common, unfortunately, even within the published literature. Dezaxa (talk) 12:02, 26 July 2012 (UTC)

"95% probability" is correct (in the frequentist framework) if you're regarding the interval as random. But if you're talking about the observed interval, given a certain sample - as we usually are in applied statistics -, then it's "95% confidence" you should say, because the probability that the parameter is within a fixed interval is always either zero or one. FilipeS (talk) 12:18, 24 February 2013 (UTC)

Sorry, Filipe, the word "random" doesn't even make sense in your statement. And No, whether to a frequentist, a Bayesian, or anyone else: A confidence interval does not represent the probability of the unknown parameter's lying in the confidence interval.Daqu (talk) 19:39, 27 February 2013 (UTC)

Filipe is right. There is a lot of misunderstanding around confidence intervals. A clear distinction has to be made between the "random confidence interval", in fact consisting of two random variabels, being the endpoints of the interval, and the realisation of this interval, the observed confidence interval, consisting of tap fixed numbers, the endpoints. The confidence coefficient is the probabilkty for the random inerval to cover the (value of the) parameter.Nijdam (talk) 23:04, 27 February 2013 (UTC)

No, Nijdam. Since the actual parameter is unknown — and the parameters have no known probability distribution — there is no way for someone to construct a confidence interval that is known to have a specified probability of containing it.Daqu (talk) 04:01, 8 March 2013 (UTC)

No, Filipe and Nijdam are correct. Treating the observed data as random (and hence the end-points of the interval as random), the probability ( or, if you want to think of the parameter as in some sense random, the conditional probability) given the true value of the parameter that the interval covers the true value is equal to the confidence level. This is true for all possible values of the "true parameter". This is essentially the definition of a confidence interval, so there can be no mistaking it. Of course, the probability here is NOT the conditional probability given the observed data. 81.98.35.149 (talk) 23:54, 8 March 2013 (UTC)

Relation to significance testing

Unless I'm mistaken, an inconsistency seems to have crept in. The second paragraph contains the sentence:

If a corresponding hypothesis test is performed, the confidence level is the complement of respective level of significance, i.e. a 95% confidence interval reflects a significance level of 0.05.

while under the hypothesis testing section there is:

It is worth noting, that the confidence interval for a parameter is not the same as the acceptance region of a test for this parameter, as is sometimes thought. The confidence interval is part of the parameter space, whereas the acceptance region is part of the sample space. For the same reason the confidence level is not the same as the complementary probability of the level of significance.

Neither of these includes a citation. It would be helpful if we could have a clarification and an authoritative citation. Dezaxa (talk) 15:48, 4 June 2013 (UTC)

Example in the "Conceptual basis" section

How would people feel if I worked on a new example for the "conceptual basis" section of the article. Currently, the example involves a measured value of a percentage. I find this confusing since the confidence level is also stated as a percentage. Maybe an alternate example with a different type of measurement could be used? Sirsparksalot (talk) 20:47, 12 September 2013 (UTC)

I am going to ask this question under my new username and see what people think (I used to edit under Sirsparksalot, now I use JCMPC). Please provide comments because I think that this is an important issue for this page. It seems as though the 90%/95% example that is given keeps flipping back and forth in a mini editing war. Personally, I feel like some of the problem is that the example given is confusing since it involves polling, which has "measured" values reported as percentages. Maybe it's my own view, but I find it easy to mix up which value is referring to the CI and which is referring to the poll results. Could we use an alternate example to help reduce some of this confusion, perhaps something such as a length measurement? JCMPC (talk) 16:29, 22 November 2013 (UTC)

I agree, please do, and if you can think of an example that would be illustrated by the bar chart that's already to the right of this section then that would be even better :-) Mmitchell10 (talk) 15:21, 21 December 2013 (UTC)

After all this time, this article contains totally erroneous statements about what a confidence interval is, and how it is computed

For example, in the introduction:

"The level of confidence of the confidence interval would indicate the probability that the confidence range captures this true population parameter given a distribution of samples."

Not at all. (And this is not the only place where this mistake is made in this article.)

This is *not* what a confidence interval means. That is surely the reason that the carefully written first paragraph does not mention anything implying that confidence level means the probability of the parameter lying in it.

It is essential that those editing this article fully understand the correct definition of the subject of this article -- or else that they step aside and make way for those who do.

Otherwise, this article will continue to misinform many thousands of readers over a period of more and more years.Daqu (talk) 19:29, 27 February 2013 (UTC)

Signing in to add a "me, too" here. The confidence interval specifically says ONLY that you can say with X% confidence that subsequent measurements will fall within this same range. It says nothing directly about whether the true value is close to the value you happened to measure. See Cellocgw (talk) 16:06, 15 August 2013 (UTC)

When computing the CI of a mean (assuming a Gaussian population), the multiplier is a critical value from the t distribution. With very large n, this converges on 1.96 for 95% confidence. For smaller n, the multiplier is higher than 1.96. This article is simply wrong. HarveyMotulsky (talk) 14:58, 14 February 2014 (UTC)

New Misunderstandings section

Please add your darlings -- always with citations. Thanks. Fgnievinski (talk) 22:36, 16 September 2014 (UTC)

Most of the quoted 'misunderstandings' are the same issue, so are they all necessary??? Also, it's not clear whether Wikipedia is saying the quotes are true statements or misunderstandings. Thanks Mmitchell10 (talk) 20:24, 17 September 2014 (UTC)

You are right. I've tried addressed some of these issues in the last edit. Fgnievinski (talk) 16:10, 18 September 2014 (UTC)

While I agree with the general thrust of the section on misunderstandings, the wording of some of the examples is not clear, and some of the references are to blog posts, which are not acceptable as authoritative sources. Also, the referenced article "The Confidence Interval of the Mean" by Oakley Gordon is not correct. He says "The second misconception is to interpret the confidence interval above as stating that there is a 95% probability that the true value of the population mean is between 46.90 and 54.10. The correct interpretation is that there is a 95% probability that the confidence interval contains the true value of the population mean." Neither of these interpretations is correct. One cannot say of a particular computed sample interval that it has a 95% probability of covering the mean, only that 95% of such samples will cover the mean in a long run of samples. I will find some time in the next few days to tidy this up unless someone else does. Dezaxa (talk) 08:32, 20 October 2014 (UTC)

I've gone ahead and rewritten the section. I hope it is clearer now. A couple of the references are still to web based sources where I would prefer a traditional text book, but many of my books are quite old. If someone can add some up-to-date text book references, then please do so. I've also taken the liberty to add the phrase "from some future experiment" to the Meaning and Interpretation section and to remove the disputed tag. I believe that with this qualification, the second paragraph of that section does not contradict what is said in the Misunderstandings section. Dezaxa (talk) 06:44, 27 October 2014 (UTC)

A quick test as to the understandability of the current article

OK, not having done statistics for ages and having just read the article, have I got this right? (Second attempt, which is a sign to me that it's not currently as clear as it might be!)

1. You do some work on a sample of a population, and you come up with a result. The 95% CI is not directly a prediction of the range of figures that the 'true' result for the whole population is in; rather it is a prediction of how accurate your process is. (Something affected by, for example, the size of your sample.) Specifically, it says that if you repeatedly sample the population and work out the 95% CI each time, then the true result will tend to be in 95% of those (very probably different albeit with plenty of overlap) confidence intervals? (In each case, it either is or it isn't...)

2. Although the 'correct' figure very probably is not exactly in the middle of the CI, in terms of being able to say 'this result is probably "about right"', the narrower the CI for any given percentage the better? Lovingboth (talk) 13:03, 28 January 2015 (UTC)

Sorry, you might want to post that to a forum such as stats.stackexchange.com Fgnievinski (talk) 01:12, 29 January 2015 (UTC)

Grumpiness, pedantry and overly long introductions

I'm approaching this page as a non-statistician and I appreciate the hard work gone into the careful crafted introduction on this page. However, as a reader I want to know from the first sentence of an article what this is and why I should care and it doesn't quite achieve this. My background is critical appraisal of medical journal articles and an accepted definition - however correct or otherwise - is "the range in which there is a 95% probability that the true value lies". I know that statisticians get a bee in their bonnet about use of the word probability in this context and get all uppity about referring to the population variable as a random statistic and to refer to this is the introduction just seems grumpy. My current favourite succint explanation is "the range within which we can be 95% confident that the true value for the population lies" . I think this or something similar should be a first sentence as it makes the whole character of the article that much more approachable and understandable. This is a common flaw of most of the statistics article where too much effort is spent being "correct" before trying to make it understandable or engaging. Arfgab (talk) 13:48, 12 April 2014 (UTC)

The problem with saying that a CI is "the range in which there is a 95% probability that the true value lies" is that this is simply incorrect. You may well be right in saying that it is an accepted definition in some circles, but it is nevertheless just a common misconception. The article's opening paragraph is not being pedantic, it is just carefully worded to avoid a common error. Exchanging this for "the range within which we can be 95% confident that the true value for the population lies" is not much better, because the word confidence invites the reader to misinterpret it as probability. The critical point is that the 95% refers to the procedure and the random intervals it produces, not to the results of any particular data set. Dezaxa (talk) 21:03, 18 April 2014 (UTC)

This isn't the place to discuss what a CI is, but rather than saying 'this is simply incorrect', I think it would be fairer to say that this is something which is disputed. Mmitchell10 (talk) 12:03, 19 April 2014 (UTC)

Dezaxa is right to say 'simply incorrect', above. For a nice nice example of how the 75% confidence interval - constructed from particular observed data - can actually include the true parameter value with probability 1, see e.g. Ziheng Yang (2006), Computational Molecular Evolution, pp. 151-152.

The CI is such a commonly used statistic that I am amazed that there is no agreement as to what it means. The page is not much help for a non-statistician and apparently not even to a statistician. Really now, what is a CI? Pcolley (talk) 22:23, 28 April 2014 (UTC)

I agree it's amazing that there is so much confusion about this, especially because the answer is very clear and has been repeatedly stated by people above: a correctly specified 95% CI should contain the true value exactly 95% of the times if repeated experiments are made (frequentist CI). It is NOT, I repeat NOT !!! "the range in which there is a 95% probability that the true value lies", regardless how much people would like to think that. The latter is called a Bayesian credible / credibility interval and has its own wikipedia page https://en.wikipedia.org/wiki/Credible_interval Incidentally, the latter also gives the correct definition for a frequentist CI, although incomprehensively written otherwise. FlorianHartig (talk) 07:04, 12 June 2014 (UTC)

Sorry, but there is no difference whatsoever between stating that an interval "contain[s] the true value exactly 95% of the times if repeated experiments are made", and stating that an interval is "the range in which there is a 95% probability that the true value lies". You're the one who is confused, about confidence intervals and about frequentism. Bayesian credible intervals are beside the point, as they arise from a different understanding of "95% probability". FilipeS (talk) 11:39, 17 July 2014 (UTC)

No, the two are quite different. One is a statement about how probable it is that a procedure will generate an interval covering a hypothesized value for a parameter, while the other is a statement about how probable it is that the parameter lies within an interval given particular data. To see how different these are, consider the following example. Suppose there is a population with some property that has an unknown mean μ, and which is distributed linearly between μ-1 and μ+1. Now suppose we are interested in calculating a 50% confidence interval. This can easily be done simply by sampling the population twice and using the two values as the ends of the interval: this works because any sampled value has a 50% probability of being above or below μ, and so there is a 50% probability that two randomly selected values will lie either side of μ. But does this mean that once you have your two values, there is a 50% chance that the interval lies inside? No. Clearly, the further apart the two values are, the more likely they are to cover the value of μ. In fact, if they are more than one unit apart, they are certain to cover μ. Conversely, if they are very close together, they are very unlikely to cover μ. The point is that conditionalizing on particular data is different from conditionalizing on unknown or future data from a procedure. Dezaxa (talk) 10:02, 16 August 2014 (UTC)

I agree with the above. Here is a less mathematical explanation. Let's say I set up traps to capture a lion in my garden. The traps are completely reliable: one trap will always activate and will always capture the lion. But before turning on my system I roll a 20-sided die. If a 7 shows up my wife will run the operation and with total certainty will sabotage the result by activating the wrong trap to save the lion. So this setup will capture the lion 95% of the time. The probability of capturing the lion using this system is 95%. I have 95% confidence in the procedure. Even if I have prior belief the lion likes to hang out near the trap by the shed the above statements are still correct.

A trap is activated. The probability the lion got captured is 95%. Next I check which trap got activated and discover it is the trap near the tree at the top of the garden. It is at this point I cannot assert that the lion is near this tree with 95% probability. It seems to me if there is only one lion and therefore I can run the experiment only once I still have a 95% confidence in the procedure. I don't have to have multiple lions. However the only probability statement I can make is the trap will work / has worked before finding out which trap was actually activated. — Axel147 (talk) 20:40, 13 October 2014 (UTC)

I must be missing something. Imagine a hat with red and blue balls. 95% of the balls are red. The probability of picking a red ball is indeed 95%. Now, imagine this hat contains the 95% confidence intervals of the sample mean from all samples of some given size "n". To be clear, all of these confidence intervals are estimated from their respective sample data and thus will differ from one another. Nevertheless, 95% of those confidence intervals contain the population mean. Any confidence interval drawn from the hat has a 95% probability of containing the true population mean. So, my one sample of size "n" will produce a 95% confidence interval of the sample mean and the probability that this confidence interval contains the true population mean will be 95%. --136.167.60.98 (talk) 18:49, 24 February 2015 (UTC)

Yes, you are missing something. Before you draw a ball, there is a 95% probability that it will be a red one. But once you've drawn it, it is either red or blue: there is no longer any probability. Even if you don't look at the ball and don't know what color you've drawn, on a frequentist understanding of probability the color of the ball in your hand is a fact about the world and there is no probability to it. A bayesian might say that there is a 95% probability that the ball in your hand is red, because to him such a statement describes his state of evidence or belief, but a frequentist cannot say this. The same is true of confidence intervals. 95% of them cover the parameter, but once you've drawn one out of the hat as it were, it either covers the parameter or it doesn't. To say that there is a 95% probability that a particular interval covers the parameter is to use bayesian language. And this is not merely a linguistic or interpretative issue: if one does want to make a bayesian statement of that kind, one would need to conditionalize on all the available information, i.e. any prior information that there might be and on the results of the experiment itself. Depending on what information is available, this might result in a probability very different from 95%. Dezaxa (talk) 13:51, 10 March 2015 (UTC)

Wrong/misleading reference

In the section "Meaning and Interpretation", there is the following definition: "Were this procedure to be repeated on multiple samples, the calculated confidence interval (which would differ for each sample) would encompass the true population parameter 90% of the time." The footnote right behind this strongly suggests that this is taken literally from "Cox D.R., Hinkley D.V. (1974) Theoretical Statistics, Chapman & Hall, p49, p209". However, if you actually take a look at the book (e.g. on Google books, where all relevant pages are freely available: https://books.google.de/books?id=ppoujo-BInsC), it turns out that this statement is not at all in the book. The footnote is therefore misleading, and I think somebody should change the article and at least make clear that the definition is not a quotation from the book, but -- at best -- a summary/rephrasing of its content.

--2003:6B:C70:8F22:A5CC:F24C:A11:A1FE (talk) 19:08, 1 August 2015 (UTC)

Utterly indecipherable to the lay-reader.

If the general public is your audience, this article is a complete failure. I'm a reader with an advanced degree, and a well-rounded education, and I can't penetrate even the lede. 66.57.50.6 (talk) 15:24, 28 September 2015 (UTC)

simple:Confidence interval? fgnievinski (talk) 16:01, 28 September 2015 (UTC)

Thank you for putting the tag. I hope someone can help so I can understand what this article is about. 66.57.50.6 (talk) 17:56, 4 October 2015 (UTC)

Missing the forest for the trees

The CI article missed or obfuscates a few critical notions:

• Confidence intervals are a way to express the mean and variability of a sample.
• Confidence implies a sample---probability has the connotation of a distribution.

As much as I like things to be Right, the right-of-way in an encyclopedia goes to the pedestrian. At least make the introduction more clear. neffk (talk) 22:19, 26 February 2016 (UTC)

Agree, & why don't you take a crack at it? Please ? --Pete Tillman (talk) 16:56, 30 March 2016 (UTC)

A few carefully chosen words word make the article much clearer

For example:

"In each of the above, the following applies: If the true value of the parameter lies outside the 90% confidence interval once it has been calculated, then an event has occurred which had a probability of 10% (or less) of happening by chance."

It took me a long time to understand why this was not an example of the common misconceptions detailed in the following section. Only when I'd understood that "an event" should be taken to mean "the action of performing sampling and calculating the specific interval" and not simply "the lying outside the interval of the population mean", did the sentence not appear at odds with a correct interpretations of the confidence interval. — Preceding unsigned comment added by 91.125.123.100 (talk) 22:37, 18 July 2016 (UTC)

I've clarified it. Loraof (talk) 21:37, 10 July 2017 (UTC)

Meaning of formula for likelihood theory

The formula for Wilk's theorem should be explained - what do the different variables stand for? — Preceding unsigned comment added by 79.177.103.37 (talk) 08:36, 6 December 2017 (UTC)

Apparent Problem in Formula

Looking at the formula $z^{*}=\Phi ^{-1}\left(1-{\frac {\alpha }{2}}\right)=-\Phi ^{-1}\left({\frac {\alpha }{2}}\right)$ presented in this article, I don't see how this can be correct. Despite the fact that I know very little statistics, I claim there is a problem here because:

$\Phi ^{-1}\left(1-{\frac {\alpha }{2}}\right)=-\Phi ^{-1}\left({\frac {\alpha }{2}}\right)$

$1-{\frac {\alpha }{2}}=-{\frac {\alpha }{2}}$

$1=0$

The formula in the article is correct, you just need to be careful of the minus sign: $\Phi \left(-\Phi ^{-1}\left({\frac {\alpha }{2}}\right)\right)\neq -\Phi \left(\Phi ^{-1}\left({\frac {\alpha }{2}}\right)\right)$ Mathandpi (talk) 23:25, 1 October 2019 (UTC)

Cleanup tag

@Botterweg14: can you list the parts you think might be inaccurate? I don't have the bandwidth to rewrite at the moment, but I can help fix inaccuracies. Wikiacc (¶) 00:28, 17 September 2020 (UTC)

Really Good link on the construction

http://personal.psu.edu/abs12//stat504/online/01b_loglike/01b_loglike_print.htm Biggerj1 (talk) 23:01, 13 July 2021 (UTC)

Needs a simple non-technical explanation to begin the article.

I think the first section of this article should more clearly and simply explain what a confidence interval is, and what motivates their use. I propose something like:

The purpose of the field of statistics is to use functions of a sample (known as a statistic) to make predictions about parameters of a sampled population. Because in general a sample will not contain all the information about a population, there will be a degree of uncertainty in predictions made, and so estimated parameters will be described by random variables rather than fixed values. A confidence interval (or CI) gives, for a specific probability, an interval such that the probability that the true value for the parameter lies in the interval, given the sample.

--Effervecent (talk) 14:35, 15 May 2020 (UTC)

Effervecent Be bold and do it. Firestar464 (talk) 12:03, 30 March 2021 (UTC)

I concur, this first section of this article is currently poor, it reads like a collection of comments and not a cohesive summary or introduction. It's far too technical, especially the latter part which should be moved into the main article. 123.255.61.246 (talk) 19:50, 21 October 2021 (UTC)

Misunderstandings Section

An anonymous editor has twice removed the text "nor that there is a 95% probability that the interval covers the population parameter" from the section under misunderstandings, with the claim that it is redundant. It is not redundant; I wish it were. There are erroneous accounts of confidence intervals which consider that it is incorrect to speak of the probability of a parameter lying within an interval but legitimate to speak of the probability of an interval covering the parameter. This is sometimes justified by saying that a parameter is a constant while the bounds of the inverval are random variables. Both statements are in fact false and it is important to state this clearly.Dezaxa (talk) 16:42, 7 March 2017 (UTC)

I don't see how you've made a distinction between the two. How is saying a given constant value is within a given range any different from saying a given range includes (or "covers") a given constant value? And where in the cited source is that distinction made? 23.242.207.48 (talk) 06:31, 10 March 2017 (UTC)

Update: I've changed "nor" to "i.e.". This way we are not giving the false impression that the two statements are distinct, but it is still clear that either way the statement is phrased it is false. This compromise should satisfy both of our concerns I think. 23.242.207.48 (talk) 21:17, 10 March 2017 (UTC)

The Misunderstandings section (and the 2nd paragraph of the summary) harp on what seems to be a pointless distinction without a difference. Of course a past sampling event either contains the true parameter, or does not. But when we speak of "confidence" that's exactly what we mean: our confidence that the sample DOES contain the parameter. To use the true-coin analogy: a person flips a coin and keeps it hidden. I am justified in having 50% confidence in my claim that the coin landed "heads". It did or did not... but confidence is referring to my level of certainty about the real, unknown value.

Now to the confidence interval: if it is correct to say that, when estimated using a particular procedure, 95 out of 100 95%-confidence intervals will contain the true parameter, than surely it must follow that I may be 95% confident that the one interval actually calculated contains that parameter. Go with the hypothetical: if the procedure was conducted 100 times, 95 of those would contain the parameter. But we have selected a subset (size one) of those 100 virtual procedures. 95 times out of 100, taking that subset will yield a sample that includes the true parameter. So I am 95% confident that this is what has happened. It either did or didn't, obviously, which is true for all statistics regarding past events. But confidence doesn't only apply to future events, but to unknown past ones.

Or one more way: I intend to do the procedure 100 times. It's expected that when I'm done, 95 of those 100 intervals will contain the true parameter. It then follows, since I have no reason to expect bias, that there's a 95% chance that the very first time I do the procedure, my interval will contain the true parameter. The fact I don't get around to doing 99 more procedures is irrelevant - I can be 95% confident that the one/first procedure performed does contain the true parameter.

How is this incorrect? (And, semi-related: I wasn't bold enough yet to edit down the pre-TOC introduction, but it unnecessarily duplicates this same Misunderstandings information). Grothmag (talk) 21:41, 6 April 2017 (UTC)

I couldn't agree more. As it stands, a "normal" person would see an instant contradiction between the meaning and interpretation section (The confidence interval can be expressed in terms of a single sample: "There is a 90% probability that the calculated confidence interval from some future experiment encompasses the true value of the population parameter.) and the misunderstanding section: "A 95% confidence interval does not mean that for a given realized interval there is a 95% probability that the population parameter lies within the interval". The only explanation currently offered is that an experiment in the past has a known outcome; it's probability is 0 or 1, and there is no doubt - the whole concept of a 95% interval is meaningless, while an experiment in the future is unknown. But that's mere word-play. The functional difference between a past experiment and a future one is merely that the past experiment corresponds to a definite, fixed population parameter (which is either in the interval or not, no shades of grey). The point about a definite, fixed population parameter is that we must know for sure whether it's in the range, so the percentage probabilities become meaningless. But (massive "but"), we never do know for sure what its value is, past or future! That's the whole point of statistical analysis. If we knew the population value, we wouldn't have to mess around calculating probabilities, we would live in a world of certainty. Since we don't know the population value in the future (because it hasn't happened yet) and we don't know it in the past (because we can't measure it) there is actually no functional difference between the two situations. Both the past and the future populations have definite, fixed population parameter values, and we are ignorant of both (albeit for different reasons).

I feel strongly that Wikipedia articles should help people understand things, rather than cover every obscure point that anyone has ever managed to get into print; we're not in the business of scoring points by being unnecessarily clever. If it's necessary to include something that is difficult to understand, then we must explain, carefully, why the point is vital, and we must endeavor to explain it appropriately for a normal reader (there is no point in writing an article that can only be understood by those who already have a thorough understanding). My feeling is that this distinction between future and past experiments requires both justification and clarification. I'm not confident enough to remove it, because it's been raised by reputable people, but if someone who knows enough could explain why the distinction is important in the interpretation of real-world experiments, the article would be greatly strengthened. 149.155.219.44 (talk) 09:08, 22 June 2017 (UTC)

In response to Grothmag and the unsigned comment above: the difference is important and is not pointless, nor is it trying to be unnecessarily clever. The crucial difference is between the probability that a method for generating intervals yields an interval that covers the parameter, and the probability that a particular realized interval covers the parameter. If I have a method for generating intervals that I can show will produce intervals that cover the parameter 95 times out of 100 in a long run of trials, then any interval that comes out of this method is a 95% confidence interval. But a particular realized interval, say 1.5 to 1.6, does not have a 95% probability of covering the parameter. It either covers it or it doesn't. Recall that confidence intervals are a frequentist concept, so its probabilities are frequencies. There cannot be such a thing as the frequency with which 1.5 to 1.6 covers the parameter, except trivially zero or one. Of course we don't know for certain whether 1.5 to 1.6 covers the parameter, but to speak of probability as a measure of how sure we are that this is true is to use Bayesian language. If you want that kind of probability you would need to calculate a Bayesian credible interval, not a frequentist confidence interval. When you say "I can be 95% confident that the one/first procedure performed does contain the true value" you seem to be using the word "confident" to mean a Bayesian probability, i.e. a degree of credibility in the truth of a proposition. A 95% probability in this sense does not follow merely from the fact that the interval comes from a method that yields covering intervals 95% of the time. This would only hold true if 1. there are no informative priors; 2. the CI is a sufficient statistic, i.e. it is capturing all the information from your experiment; and 3. there are no nuisance parameters. These are the conditions under which a confidence interval and a credible interval coincide. Dezaxa (talk) 11:58, 26 September 2017 (UTC)

I have to say, I have been mulling over this point, and it does seem like the way it is currently explained, there appears to be a distinction without a difference. I don't think that's actually the case but it would help if there was some example that would illustrate the distinction....the only example I can think of is that I can understand the error if you erroneously think the confidence interval represents a range of values that covers 95% of some probabilistic distribution of the true parameter. So an error in reasoning would be to think that by getting a particular confidence interval, that by choosing any of those values inside it as the true parameter, you are 95% chance within x/2 (where x is the length of the CI) of the true parameter. In actuality, you do not know how close or far any value is from the true parameter from only a single confidence interval calculation, because the true parameter has no distribution. You could be a little bit off or you could be way off. Or, more generally, you could make a mistake of thinking that any values within your confidence interval were somehow useful, when in fact they were meaningless.

But the way it's explained it doesn't answer the following question: Take the statement "The true value is an element of the set of values contained in the confidence interval." This will be correct 95% of time for a 95% confidence interval. If you were making a bet based on that statement, you would still win the bet 95% of the time wouldn't you? So why shouldn't one think of this as a probability? Why is this wrong (if it is wrong)? If it's not wrong, why is that different from saying the probability that the true value is inside the confidence interval is 95%? What's an example that would show these? 108.29.37.131 (talk) 21:24, 14 June 2018 (UTC)

I appreciate your efforts to explain, Dezaxa, and I wish I had noticed them earlier to respond in a more timely fashion. I remain unconvinced, partly because your response runs counter to this quote from Wikipedia's entry on "frequentist probability": "An event is defined as a particular subset of the sample space to be considered. For any given event, only one of two possibilities may hold: it occurs or it does not. The relative frequency of occurrence of an event, observed in a number of repetitions of the experiment, is a measure of the probability of that event". Note that a (non zero or one) probability still exists for the event, since probability is a measure of the sample space, not of the event.

Therefore we can still speak of the probability of an event, regardless of the fact it has occurred or has not (the two possibilities). The event had a particular probability in sample space... in this case, the true parameter falling within the "confidence interval" of the interval-generating method is the event in question. As you said yourself, the confidence interval is a frequentist concept. If the method generates intervals including the parameter 95% of the time, then any particular interval, regardless of whether it contains the parameter or not, had a 95% probability of doing so. This is where my objection arises, and where the gambling analogy of 108.29.37.131 fits nicely. This is a probability, in one sense of probability. If we are correct in our confidence interval, than it really should give us "confidence" ... We should be able to say, with 95% or whatever "odds" of being correct, that the true parameter falls within the interval generated by our method. I really don't see how this statement can be incorrect unless our method itself is flawed, in which case our confidence intervals are themselves incorrect (as they would be if nuisance parameters were ignored). Grothmag (talk) 23:27, 12 September 2018 (UTC)

I can only repeat that there is a crucial difference between speaking of probability as the long-run frequency of a hypothetical set of trials and the 'degree of confidence' attaching to a particular, observed instance of a trial. Once a trial has taken place and the result is known, this provides new information that must be conditionalised upon when assessing the probability. If my explanation is not clear enough, you might care to consult the paper referenced in the article: Morey, R. D.; Hoekstra, R.; Rouder, J. N.; Lee, M. D.; Wagenmakers, E.-J. (2016). "The Fallacy of Placing Confidence in Confidence Intervals". Psychonomic Bulletin & Review. 23 (1): 103–123. doi:10.3758/s13423-015-0947-8. PMC 4742505. PMID 26450628. It is worth noting also that the section "Counterexamples - confidence procedure for uniform location" provides an example of how an interval can be a 50% CI and yet not have a 50% probability of covering the parameter. It is really an instance of the same issue. Dezaxa (talk) 05:55, 30 November 2018 (UTC)

Dezaxa, thank you. The counter-example in the article is, indeed, a very good demonstration of your point, and though I did check out and enjoy Morey et al. too, that was really just the icing on the cake - I'm convinced. The only thing I might still stick on is the article's statement "once an interval is calculated, this interval either covers the parameter value or it does not; it is no longer a matter of probability" - since (as the Frequentist Probability article indicates) one can still discuss the probability that the interval includes the (presumably still unknown) parameter - events can have (had) probabilities after the fact. One just can't naively call it 95% (or whatever value arising from the confidence procedure). But that's a different issue, and I'll let it go. Grothmag (talk) 01:32, 4 December 2018 (UTC)

Does anyone still want a simple, everyday example for a (say 95%) confidence interval not being the same as there being a 95% probability of the parameter being contained within it? Consider using Google Maps (other mapping software is available) on your phone. You might use it dozens of times per week and most of the time the circle it draws contains your actual position. Sometimes, perhaps you have emerged from an underground public transport system, it draws a small circle but you are many miles (other distance units are available) away. Get off a plane and it might say you were still at the departure city. If Google correctly encompasses you 95% of the occasions you use the app it can be regarded as a frequentist experiment with your position as the parameter being estimated. The mapping circles are genuine 95% confidence intervals as they will truly contain the parameter value (where you actually are) 95% of the times you run the experiment (use the app). I hope it's clear that for any single use of the app you are either correctly placed inside the circle (i.e. it is 100% probable that you are within the circle) or you are outside ( = 0% probable that your position is within the circle). Many people assert that a 95% confidence interval means they can be "95% confident" about a single interval. In this example the app user would be closer to the correct meaning if they said "95% of the time Google is spot on but 5% of the time it is completely wrong". The "confidence" refers to how often the experimental method (such as the mapping application) correctly encompasses the parameter value. — Preceding unsigned comment added by 82.0.253.251 (talk) 18:53, 11 December 2019 (UTC)

There is yet another example and explanation in: https://arxiv.org/abs/1411.5018 (pp. 4-6) Spidermario (talk) 23:04, 10 August 2021 (UTC)

I just wanted to insert a reiteration of Dezaxa's first point, because it helps me keep this straight. The CI is a realization of a Random Variable, the bounds both being themselves functions of the data. The parameter is a constant. It is nonsensical to speak of a constant having a probability of lying in some interval. An analogy would be: suppose you are flipping a fair coin. It has a 50% chance of being heads or tails. Suppose you flip the coin and it lands on heads. There is not a 50% chance that the coin is tails.Nathan.s.chappell (talk) 07:46, 19 August 2021 (UTC)

Nathan, I do not think analogy quite works. If I flip a coin but do not look at whether it is heads or tails, then I could still say there is a 50 percent chance that the coin is tails. When I calculate an interval, I do not get to see the parameter so I am not like the person who sees the coin, rather I am like the person who has flipped the coin, but still has the coin covered. As mentioned earlier in the thread, the whole issue is whether it is reasonable to make unconditional probability statements after further facts are known. Complaints about the parameter being a constant are purely technical. Consider that the source of the randomness (the interval or the parameter) does not impact the "confidence" we have basing decisions off the interval. It is not that the parameter is constant, but that there is a better probability (or confidence) statement that can be derived using all the information in the sample rather than the information known before the sample was observed. In this way the coin example is apt, but extreme. In the coin example you have all of the information after you observe the coin, whereas, in the confidence interval example you might only have a small amount of additional information to condition on (beyond what is accounted for in the interval) when you have the sample. 161.203.19.1 (talk) 04:00, 18 December 2021 (UTC)