Talk:Bayesian probability/Archive 3

This is an archive of past discussions about Bayesian probability. Do not edit the contents of this page. If you wish to start a new discussion or revive an old one, please do so on the current talk page.

Archive 1

Archive 2

Archive 3

Jaynes's Errors

This article fails to warn that Jaynes machinery often fails to provide priors (non-existence) or provides non-probability measures (improper). Kiefer.Wolfowitz (talk) 12:02, 14 June 2009 (UTC)

Fine. So add some stuff about it, but please provide some decent references for a change. Tomixdf (talk) 13:18, 14 June 2009 (UTC)

Tommixdf should cool his tone, especially since he has managed to bias this article despite years of objections from statisticians. This article has the quality of an article on objectivism or scientology.Kiefer.Wolfowitz (talk) 13:26, 14 June 2009 (UTC)

I'm asking you again, very politely, to work towards compromise instead of conflict. In the case of the Dutch book argument for example, we reached a conclusion that everybody was happy with and that actually made the article better. Tomixdf (talk) 13:34, 14 June 2009 (UTC)

Okay. Sorry for irritation, partly from your previous comments but mostly from pseudofed and a fever.Kiefer.Wolfowitz (talk) 13:53, 14 June 2009 (UTC)

Jaynes certainly made his share of mistakes, but don't blame him for errors in this article. Is it Jaynes' "mistake" if Jaynes' methods lead to the use of an improper prior and Jaynes himself says explicitly that it's improper? Michael Hardy (talk) 16:19, 14 June 2009 (UTC)

Well said! I would rephrase the title of this section as "Errors of Jaynes and Jaynesian enthusiasts" but have been repeatedly cautioned against editing the history of talk pages! Kiefer.Wolfowitz (talk) 16:31, 14 June 2009 (UTC)

Inconsistencies of Frequentist Statistics:

Tomixdf removed "alleged" (first) and "perceived" (latest) before "inconsistencies of frequentist statistics", both times labelling these adjectives as "weasel words". The article fails to identify or explain any "inconsistencies". Please explain before putting them back in the article. Kiefer.Wolfowitz (talk) 10:38, 18 June 2009 (UTC)

Agreed. Will provide decent reference. Tomixdf (talk) 15:39, 18 June 2009 (UTC)

The probability statements of frequentist procedures refer to properties of the procedures when used on a population of such problems (not for a particular sample). Of course, non-statisticians have trouble interpretting confidence statements (for realizations of a random variable); of course, trying to interpret "confidence" as a probability will lead to trouble---But such mistakes are not contradictions!)Kiefer.Wolfowitz (talk) 16:07, 18 June 2009 (UTC)

Objections to Laplace's "Objective" priors (by Fisher and others)

All of the criticisms of Laplace (with which I am familiar) object to "objective" priors (Principle of insufficient reason).

Laplace also used non-uniform priors sometimes. Can anybody find criticisms of all uses of "non-objective" priors by Fisher?

[Fisher's words are reverently quoted (despite the notorious unreliability of statistician's "history" and especially Fisher's scholarship, sic.) but Fisher's quoted objections seem to apply to what Fisher (with characteristic obscurity) called "axiomatic priors"?]

Should not mention be made of Laplace's informed priors? (If agreeable, then I can look at Hald's serious volume through 1930, or Stigler).

(These questions are indeed part of my "agenda" to reduce the false dichotomy of objective and subjective Bayes methods, I confess!) Kiefer.Wolfowitz (talk) 14:17, 21 June 2009 (UTC)

There is no "false dichotomy": there are many references that discuss and oppose these two versions of Bayesian probability (see many Jaynes references for example). That difference may not be there (according to you that is indeed the case, it seems), but the references surely are there. What can be done, is to find a reference that clearly states that this difference is a "false dicotomy" or similar according to someone. Original research will not do! Tomixdf (talk) 14:59, 21 June 2009 (UTC)

What do you think of the substantive & editorical questions? (I gave a parenthetical confession to avoid being "outed" again---or at least to save you typing . . . . )Kiefer.Wolfowitz (talk) 15:24, 21 June 2009 (UTC)

Here's a fine example. Gelman wrote a provocative article, in which a hypothetical anti-Bayesian argues against Bayesian methods. Bernardo answers, stating that he answers from the "objective Bayesian" POV. Joseph Kadane states he answers from the subjective POV. So false or not, the distinction is out there. I don't know what you mean by "substantive & editorical questions", "parenthetical confession" or "outed". Tomixdf (talk) 07:47, 22 June 2009 (UTC)

I repeat my questions:

Can anybody find criticisms of all uses of "non-objective" priors by Fisher? (Again, Fisher used informed priors.)
Should not mention be made of Laplace's informed priors?

Kiefer.Wolfowitz (talk) 12:06, 22 June 2009 (UTC)

What exactly is your point? Tomixdf (talk) 12:17, 22 June 2009 (UTC)

Empirical Bayes

This article cites Turing, Good, etc. as being Bayesian.

Please do not remove statements about "Empirical Bayes" here without substantial justification; such justification could entail removing Turing and Good also. Kiefer.Wolfowitz (talk) 18:47, 21 June 2009 (UTC)

This article is about Bayesian probability.

Good and Turing belong in this article because they clearly identified their philosophical view of probability as classical/Bayesian; not because they may sometimes have used Empirical Bayes.

"Empirical Bayes" isn't a method based on Bayesian probabilities. It's a method based on frequentist probabilities. (Which is why Neyman liked it). Yes, it can be regarded as an approximation to a fully Bayesian method. But along the way everything which requires a distinctively Bayesian view of probability is dropped.

It doesn't require a Bayesian "degree of belief" interpretation of the hidden parameters θ. Rather, the θs are seen as a number of samples from a large population - a fully frequentist conception, allowing them to be seen as frequentist random variables. As for the hyperparameters φ, a Bayesian would draw up priors for them. But EB doesn't. Instead it treats them as frequentist parameters of the distribution of the random variables θ, and estimates them in a frequentist think-of-an-estimator way.

So that's why, if you look at eg the Gelman Bayesian Analysis "Objections" paper and its follow-ups, it's quite amusing just how at pains the Bayesians are to emphasise EB not being a "Bayesian" method. Jheald (talk) 19:48, 21 June 2009 (UTC)

I agree with most of what you write, and we could have both found similar citations from Dennis Lindley, who has written that Empirical Bayes is the non-Bayesian method around! (I have been told.)

Nonetheless, Good and others have used EB as part of the Bayes/Berkeley compromise, which is necessary in statistical practice. Who can be a subjectivist Bayes with more than a few parameters? But Good is a much more serious statistician and philosopher than Gelman (who writes about "noninformative priors" without embarrassment), and so I would ask that we take Good at his own word, that EB is a useful Bayesian method. (I trust that I don't need to find a quote from Good Thinking!) Kiefer.Wolfowitz (talk) 20:21, 21 June 2009 (UTC)

The point, though, is that it doesn't require any belief in "Bayesian probability" to use and believe in EB. So it is misleading if the article could be construed to imply, even potentially, that a readiness to accept EB suggests any support for the Bayesian idea of probability. Jheald (talk) 20:39, 21 June 2009 (UTC)

Does not your Empirical Bayes warn the faithful against the infidels, Savage and de Finetti? Is that not warning enough? (Smiley face)

What I wrote was that Neyman used (the?) Bayesian formalism repeatedly, and considered EB great. Delete the link to EB at the bottom of the page (under see also), if you want, but don't touch my lines on Neyman! Please! Is that okay? Kiefer.Wolfowitz (talk) 20:50, 21 June 2009 (UTC)

But this article isn't about the Bayesian formalism. That has its own article. This article is about the Bayesian view of what probability is. Jheald (talk) 21:14, 21 June 2009 (UTC)

I agree with Jheald. What is the point of mentioning that Neyman was fond of EB (a frequentist method)? Or that Fisher and Neyman sporadically used methods that could be viewed as Bayesian? The point is that they both promoted and adhered to a very different view on probability. Kiefer.Wolfowitz's contributions only serve to obscure this fact. Tomixdf (talk) 06:16, 22 June 2009 (UTC)

In the Neyman reference that was provided in the EB sentence, Neyman writes "A brilliant idea as to how it can be done, initiating a novel chapter of frequentist mathematical statistics, is due to Herbert Robbins." That settles it, surely: Neyman saw EB as frequentist (which it also is). I removed the EB sentence. Tomixdf (talk) 13:48, 22 June 2009 (UTC)

Neyman as Bayesian

The article states: "Neyman also used Bayesian methods in teaching[12] and sometimes in research[13]." I glanced through [12], and it is a firm defense of the frequentist view. There is no mentioning of Neyman using Bayesian methods. [13] is a technical paper by Neyman; thus the conclusion that "Neyman also used Bayesian methods in research" is Original Research, and not up to Wikipedia standards. If no better references can be provided, the sentence needs to be removed. Tomixdf (talk) 14:00, 22 June 2009 (UTC)

I'll rephrase the disputed sentence later. However, your objectionable heading for this section misrepresents my sentence, about Neyman's use of Bayesian methods.

The article (12) ends (from memory) with explaining how Churchill suggested using the sampling distribution of an estimator, after Neyman taught Bayes with a uniform prior, so your synopsis of 12 is wrong. By Tarski's convention T, the sentence "Neyman used Bayesian methods in research" is true if Neyman used Bayesian methods in research; hence the relevance of [13], which is technical (like most statistical articles, you might as well learn now). Kiefer.Wolfowitz (talk) 14:41, 22 June 2009 (UTC)

(12) ends with Neyman thanking Churchill Eisenhart for suggesting that "the whole theory would be nicer if it were built from the start without any reference to Bayesianism and priors". He then writes "This remark proved inspiring". (Not from memory; the article is in front of me) That is exactly the opposite of what you try to use this reference for. Tomixdf (talk) 14:54, 22 June 2009 (UTC)

Tomixdf, please confirm that Neyman describes teaching inverse probability (Bayesian methods) in that article (which was the proposition I wrote, and cited that article as the most convenient and well known example).Kiefer.Wolfowitz (talk) 15:57, 22 June 2009 (UTC)

The article does NOT state "Neyman uses inverse probability (Bayesian methods) in teaching". Please stop abusing references to push your POVs. Tomixdf (talk) 16:01, 22 June 2009 (UTC)

QUOTE: "Neyman also used Bayesian methods in teaching" is what I wrote. What is your problem (here)? Please apologize. Kiefer.Wolfowitz (talk) 16:08, 22 June 2009 (UTC)

My problem is the CONTINUOUS abuse of references to push your POV. I dare you to cite a section in the article that states that "Neyman also used Bayesian methods in teaching". Tomixdf (talk) 16:11, 22 June 2009 (UTC)

This is ridiculous. Neyman differs from the Queen of England in not referring to himself in the third person. Your "dare" is irrelevant. Kiefer.Wolfowitz (talk) 16:16, 22 June 2009 (UTC)

Fine. Then provide a section that states "I also used Bayesian methods in teaching". 16:19, 22 June 2009 (UTC)

I know that you are an inexperienced Wikipedia editor (judged by your removing text on the talk page, including jokes in articles, personal insults and innuendo, spamming the talk page), so please take the time to read up on original research. You need a reference that specifically states "Person A is a Bayesian", not "I think Person A is a Bayesian because something in article X by Person A makes me think so". That is OR and not acceptable in Wikipedia. Tomixdf (talk) 15:02, 22 June 2009 (UTC)

I never included a joke in the article. What innuendo are you talking about? Kiefer.Wolfowitz (talk) 15:57, 22 June 2009 (UTC)

You included a link to "problem gambling" as a joke, and even tried to defend it afterwards. Beginners make mistakes, and that's understandable, but at a certain point it should start to improve (quod non). Tomixdf (talk) 16:14, 22 June 2009 (UTC)

That was not a joke, although it was funny imho. Why don't you write van Fraassen and ask him to assess the text I wrote? Kiefer.Wolfowitz (talk) 16:19, 22 June 2009 (UTC)

Why on earth would I do that? You need to read up on Wikipedia practice. Tomixdf (talk) 16:22, 22 June 2009 (UTC)

Scientific Method

This section seems a little biased to a non-Bayesian like me. It says that scientific practice (or good scientific practice) can be interpreted as an application of Bayesianism. Surely this is controversial. It also implied that the criticisms of Bayesianism would warrant further examination if there were concrete examples of Bayesianism going wrong. Again, this seems too much of an opinion for an encyclopedia entry. One might think that the problems are as serious in the abstract as they are in the concrete. I've at least corrected this, so that it reads 'might' rather than 'would.'

It now reads that scientific method "is sometimes interpreted" as an application of Bayesian updating---the existing footnote refers to two standard sources. Kiefer.Wolfowitz (talk) 18:41, 17 September 2009 (UTC)

I propose adding a link in external references

The proposed link is http://www.opentradingsystem.com/quantNotes/Conditional_probability_.html

It contains examples of calculations using Bayesian formula.

Does anyone object? —Preceding unsigned comment added by Kaslanidi (talk • contribs) 20:24, 30 December 2009 (UTC)

I object, per WP:ELNO points 1, 4, and 11. - MrOllie (talk) 20:31, 30 December 2009 (UTC)

I second the objection. OhNoitsJamie ^Talk 20:42, 30 December 2009 (UTC)

Move proposal

I think that some expansions of this article have introduced overlap and duplication with other articles on Bayesian stuff, moving away from the main topic of this article which is supposedly the Bayesian interpretation of probability. I propose that the section headed "Scientific method" should more properly be moved to Bayesian inference. There are other parts with extreme overlap with that article, such as that on updating of beliefs. Any thoughts? Melcombe (talk) 10:43, 18 February 2010 (UTC)

I disagree. The article on Bayesian inference seems to have a very practical angle. Surely "Scientific method" fits more in this article than in the other one. Tomixdf (talk) 11:19, 18 February 2010 (UTC)

But what is being said is more about Bayesian inference that Bayesian interpretation of probability. The lead section of this article and the article's placement in Category:Probability interpretations suggests that "Bayesian interpretation of probability" should be the focus here, making distictions from other interpretations. Melcombe (talk) 11:42, 18 February 2010 (UTC)

Logic

I do not see much logic in the way this article is arranged. The title is "Bayesian probability", and a section is called "Bayesian probability calculus". What the difference may be, I wonder. Furthermore, Bayes' formula is just a theorem in (general) probability (calculus) and not especially in Bayesian theory, and hence "Bayes" formula" is not the main article of the section called Bayesian probability calculus. Neither is Bayesian inference the main article to this section. Nijdam (talk) 16:24, 31 August 2010 (UTC)

Okay, it is much better now. Nijdam (talk) 14:07, 1 September 2010 (UTC)

Subjective probability

I do not know whether all subjective probability is termed Baysian. Nijdam (talk) 08:31, 11 October 2011 (UTC)

Interpretation

The excerpt: 'Bayesian probability interprets the concept of probability as "a probability p is..."' is controversial. Reasons:

stylistically, it does not look well written (the citation does not fit well into the sentence citing it)
the cited text was written by E.T. Jaynes. Other Bayesians, such as de Finetti strongly disagree with it, thus, it is not appropriate as a general characterization
even Jaynes in his later works disagrees with it, writing that the purpose of assigning probabilities is actually the "plausible reasoning", not the "representing the state of knowledge"

Thus, the general characterization should either be replaced by a truly general one, or, at least, it should be mentioned that the characterization is not general, and applies only to a particular POV Ladislav Mecir (talk) 00:29, 27 December 2011 (UTC)

Bayesian Frequentists

In the modern world, Bayesian and frequentist "interpretations" aren't diametrically opposed, and there is no need to consider oneself "a Bayesian" or "a frequentist". A Bayesian surely will not dispute that prior information is often obtained, whether directly or indirectly, with input from frequencies, and will surely not claim anything is wrong with the frequentist approach in the absence of additional prior information. Likewise, modern frequentists will happily use Bayesian probability so long as the conditional distributions lead to an ability to perform repeatable trials when such conditions are in place. — Preceding unsigned comment added by 149.97.32.36 (talk) 00:15, 11 January 2012 (UTC)

Are they said to be "diametrically opposed" in the article? There is a difference between them. This article needs to be reasonably clear about what the various varieties of Bayeasian-type probabilities are (how they areinterpreted). How they are used (and how their use differs/compares to frequentist and Neyman arguments) is better placed in Bayesian inference or statistical inference, or possibly in probability interpretations, or in a new article specifically for such philosophical matters. It may already be there to some extent. Melcombe (talk) 09:11, 11 January 2012 (UTC)

Epistemic probability

"Epistemic probability" redirects here, but the term is never mentioned. This is always unsatisfactory since it leaves one unsure whether "Epistemic probability" is exactly synonymous with "Bayesian probability" or is just a related concept. 86.160.212.146 (talk) 13:37, 18 July 2012 (UTC)

common applications

Conspicuously absent is any section describing common applications where Bayesian probability is useful. Email spam filters are an excellent example. 148.87.19.218 (talk) 16:48, 11 October 2012 (UTC)

Love how much intelligence goes into Wiki, but hate how often it seems to assume a college education or prior knowledge of a topic. Knowledge in this context is only useful when understood by others. Can you explain it so a first grader can understand it, and if not, have you tried? Thanks. — Preceding unsigned comment added by 99.23.77.182 (talk) 17:03, 15 November 2012 (UTC)

Concern of plagiarism for part of the text

The following excerpt

"It is known that Cox's 1961 development (mainly copied by Jaynes) is non-rigorous, and in fact a counterexample has been found by Halpern.[16] The assumption of differentiability or even continuity is questionable since the Boolean algebra of statements may only be finite.[7]"

has been copy-pasted from "New Axioms for Rigorous Bayesian Probability", Dupre, Tipler. http://ba.stat.cmu.edu/journal/2009/vol04/issue03/dupre.pdf

It seems like this section is just rephrasing poorly the article anyway.

I don't know too much about wikipedia etiquette for such situations, but that does seem pretty bad. Why not cite Dupre, Tipler explicitly instead ?

Also, this section doesn't make it clear that Dupre, Tipler offer a (seemingly) rigorous axiomatic construction — Preceding unsigned comment added by 2001:620:600:6000:8012:59CD:941C:60BD (talk) 14:16, 7 August 2015 (UTC)

This excerpt is no longer present in the current version. Archiving. — Gamall Wednesday Ida (t · c) 19:44, 9 February 2017 (UTC)

Dr. Koop's comment on this article

Dr. Koop has reviewed this Wikipedia page, and provided us with the following comments to improve its quality:

Excellent, although the very last bit ("Bayesian average") seems out of place. The article could easily end just before this. Right now it does from being a fine general discussion of Bayesian probability and its uses to a very specific concept (using terminology I have not heard before). Dale Poirier at the University of California, Irvine would be a good choice to get feedback on this article

We hope Wikipedians on this talk page can take advantage of these comments and improve the quality of the article accordingly.

Dr. Koop has published scholarly research which seems to be relevant to this Wikipedia article:

Reference 1: Joshua C C Chan & Eric Eisenstat & Gary Koop, 2014. "Large Bayesian VARMAs," Working Papers 1409, University of Strathclyde Business School, Department of Economics.

Reference 2: Koop, Gary & Korobilis, Dimitris, 2014. "Model Uncertainty in Panel Vector Autoregressive Models," MPRA Paper 58131, University Library of Munich, Germany.

ExpertIdeas (talk) 11:29, 18 July 2015 (UTC)

Support Thanks for the ExpertIdeas. I support ending the article before the "Bayesian average" section as proposed. It indeed looks out of place, not contributing to the general discussion of Bayesian probability. Ladislav Mecir (talk) 05:35, 19 July 2015 (UTC)

Support removal of Bayesian average section. It is a small unreferenced section. It was possibly added to serve as an example of Bayesian probability reasoning, but doesn't help in that task and seems out of place otherwise. And thanks for the expert review. --Mark viking (talk) 22:57, 20 July 2015 (UTC)

Dear @Ladislav Mecir and @Mark viking, I am the developer of ExpertIdeasBot. Thank you so much for your support. We'll appreciate it if you take a look at our list of edit requests by economists and help us by incorporating them into the Wikipedia articles. Both economists whose publications have been highly cited and skillful Wikipedians with expertise in organizing knowledge on Wikipedia are powerful resources that we are trying to connect in order to improve the accuracy and usefulness of Wikipedia content. Please let us know about your ideas and proposals to improve our service. I.yeckehzaare (talk) 20:06, 28 May 2016 (UTC)

External links modified

Hello fellow Wikipedians,

I have just added archive links to 4 external links on Bayesian probability. Please take a moment to review my edit. If necessary, add {{cbignore}} after the link to keep me from modifying it. Alternatively, you can add {{nobots|deny=InternetArchiveBot}} to keep me off the page altogether. I made the following changes:

When you have finished reviewing my changes, please set the checked parameter below to true or failed to let others know (documentation at {{Sourcecheck}}).

Y An editor has reviewed this edit and fixed any errors that were found.

If you have discovered URLs which were erroneously considered dead by the bot, you can report them with this tool.
If you found an error with any archives or the URLs themselves, you can fix them with this tool.

Cheers.—^{cyberbot II}_{Talk to my owner:Online} 13:34, 22 March 2016 (UTC)

Readability for non-experts

This article is a perfect example of something I run into all the time on Wikipedia: technical articles are written by experts in the field and lack an explanation that is accessible to non-experts. Consider adding a paragraph at the beginning which explains the concept in simple, easy-to-understand terms.

I know that technical folks are often loathe to do this, because they are afraid of losing accuracy if they try to simplify things. If technical terms can't be avoided, they need references so that a non-expert can at least look them up. The first paragraph alone of this article would require links for "evidential probabilities", "extension" (of logic), "reasoning", "propositions", "truth or falsity", "uncertain", "evaluate", and "probabilist". But generally it would be way more pleasant if Wikipedia article started simple and got more complex as you read down. — Preceding unsigned comment added by Kunstcleaver (talk • contribs) 15:40, 8 February 2012 (UTC)

I strongly agree that this needs an introductory paragraph from a lay person's perspective. Ceolas (talk) 22:27, 2 April 2012 (UTC)
I totally agree, even though I've taken a course in statistical maths. Please someone, try to explain the essence in understandable terms. Physicists are really good at this, but the math's articles at Wikipedia are generally too inaccessible to most people. --81.216.218.158 (talk) 22:01, 10 November 2012 (UTC)
Agree with all of the above (including the comment about physicists). I also took a course in stats as part of my physics, and find this article (and many other maths articles on Wiki) very obscurely written.

More than one year after these comments: The article is still at the same point. I've done quite a lot in statistics, on a professional level, as a "frequentist", however. After reading this article, I have no idea what Bayesian probability is about, except that it claims to be better than all those frequentists do... Come on, technical folks, you can do better!--Panda17 (talk) 08:03, 8 March 2014 (UTC)

C'mon, Panda, you can type! GcT (talk) 10:13, 22 January 2015 (UTC)
I edited the introduction with a reference from a mid-century Encyclopædia Britannica. Please do not restore this without a "Discussion" as there are five comments above that agree with having an intro such as I wrote.

There were no references in the old introduction; a bad thing. I moved the old intro contents into their own section "Interpretation". The old introduction was a discussion of the Philosophy of Probability, which is a separate article in Encyclopædia Britannica and should be here in Wikipedia. I feel this topic has no place in this article, but I left it as I found it, moving one paragraph to my new intro. Nick Beeson (talk) 11:48, 5 July 2015 (UTC)}

Oppose The cited text is incorrect and obsolete. For example,

"Bayesian probability provides a way of calculating the most likely probability" - that is certainly false. First of all, Bayesian probability can be either objective or subjective, and the goal of the subjective Bayesian probability is certainly not to calculate "the most likely probability", but, instead, it is to find the probability that corresponds to a personal belief constrained by the requirements of rationality and coherence. However, even for the objectivist, the goal is not to find "the most likely probability", but to find the probability that objectively represents the given state of knowledge. Also, the text pretends to be simple, but it does not define the notion of "the most likely probability".

"It gives an estimate of the probability of event n happening given that, in the past, m events happened for every n events. The events must be independent, and at the start any value of the probability between 0 and 1 is equally likely." - this is also incorrect. First of all, it is not true that Bayesian probability can handle only independent events. Bayesian probability can handle more general cases than the described one. Also, Bayesian probability can handle other priors than the uniform distribution. Ladislav Mecir (talk) 18:35, 5 July 2015 (UTC)

Nwbeeson Please note that many of the above objections were addressed inbetween. Ladislav Mecir (talk) 18:40, 5 July 2015 (UTC)