Wikipedia:Reference desk/Archives/Mathematics/2023 July 12

Welcome to the Wikipedia Mathematics Reference Desk Archives
The page you are currently viewing is a transcluded archive page. While you can leave answers for any questions shown below, please ask new questions on one of the current reference desk pages.

July 12

Axioms of probability

Why exactly isn't the definition of independence as $\mathrm {P} (A\cap B)=\mathrm {P} (A)\cdot \mathrm {P} (B)$ regarded as an axiom in fact? Hildeoc (talk) 13:47, 12 July 2023 (UTC)

In an axiom of logic like $A\to (B\to A)$ you can substitute any propositions for the variables $A$ and $B$, and get a valid proposition, also if you use the same proposition twice. So, for instance, both $2{+}2\,{=}\,4\to (2{+}2\,{=}\,4\to 2{+}2\,{=}\,4)$ and $2{+}2\,{=}\,5\to (2{+}2\,{=}\,5\to 2{+}2\,{=}\,5)$ are valid propositions. Now if we substitute the same event $X$ for $A$ and $B$ in the proposed axiom, we find that $\mathrm {P} (X)=\mathrm {P} (X\cap X)=\mathrm {P} (X)\cdot \mathrm {P} (X)=\mathrm {P} (X)^{2},$ implying that $X$ is either almost certain or almost impossible. To avoid this conclusion and save the axiom, we need to require that the events $A$ and $B$ in the axiom are independent. So we need another definition. One could use the definition $P(A|B)=P(A),$ but then – assuming that $P(A|B)$ is defined as ${P(A\cap B)} \over {P(B)}$ – the axiom becomes a theorem. However, this definition fails when $P(B)=0$; moreover, we have lost the symmetry between $A$ and $B$, so this is not an improvement.  --Lambiam 17:16, 12 July 2023 (UTC)

Not quite sure that's what Hildeoc was getting at.

Of course, at a trivial level, any definition of a predicate (like "independent") can be formally rephrased as an axiom, by adding the predicate as an undefined term, and then adding an axiom that says the predicate is true if and only if (substitute definition here). But that's probably not what Hildeoc meant either, because it's too trivial.

Another possibility is that Hildeoc is relying on some unformalized notion of independence (say, events A and B do not influence one another causally and are not influenced by a common cause). That notion is stronger than what can be captured by probability theory.

For example, say we throw a coin a countably infinite number of times, and the throws are independent (not just pairwise, but altogether) in this informal sense. Then the probability that they all come up heads is zero; nevertheless, the outcome is possible. That's because any throw possibly comes up heads, so to exclude the possibility that they all come up heads would require some dependency among the throws.

The event that all odd-numbered throws come up heads is independent of the event that all even-numbered throws come up tails, both in the informal sense and in the probability-theory sense, the latter because 0×0 = 0.

But the event that all throws come up heads is independent in the probability-theory sense from the event that all throws come up tails (again because 0×0 = 0). But the two events are not independent in the informal sense, because their conjunction is impossible. --Trovatore (talk) 22:15, 12 July 2023 (UTC)

I thought of a possibly more accessible way of showing that probabilistic independence doesn't capture informal independence.

Say I have two coins, which are both fair individually. However they are linked in the following unusual way: If I throw them both before noon on a given day, they will always come up either both heads or both tails. If I throw them after noon, they will always come up opposite from one another.

Now I choose a random time and throw both coins at that time.

You can easily check that the throws are independent by the probability-theory definition. Each coin comes up heads 50% of the time and tails 50% of the time, and the probability of each combination is 25%; the arithmetic all works.

But I think it's clear that the throws are not "really" independent. They have two different dependencies on one another, which cancel out probabilistically. --Trovatore (talk) 22:22, 13 July 2023 (UTC)

@Lambiam, Trovatore: I was actually referring to the three axioms of probability. Did that come across? When using $\mathrm {P} (A\cap B)=\mathrm {P} (A)\cdot \mathrm {P} (B)$ as a definition, or likewise, the definition of conditional probability (as applied to the multiplication theorem, which can be inferred from the former), why exactly do those invoked issues with $A$ and $B$ both becoming $X$, or ${P}(B)=0$ apparently not occur? And, in conclusion, what's the conceptual difference leading to calling this merely a definition and not an axiom – as opposed to the additivity axiom $P(A\cup B)=P(A)+P(B)$ for disjoint events? Hildeoc (talk) 23:16, 15 July 2023 (UTC)

It is an issue of terminological convention. When dealing with a mathematical structure (a set endowed with some additional features on the set, such as a metric), it is conventional to use the term "axiom" for the defining properties of the additional features of that particular kind of structure. For example, in Metric space § Definition we see:

Formally, a metric space is an ordered pair (M, d) where M is a set and d is a metric on M, i.e., a function $d\,\colon M\times M\to \mathbb {R}$ satisfying the following axioms for all points $x,y,z\in M$: ...

These axioms capture what is special about metric spaces. In the same article we find a definition of distance-preserving function:

A function $f:M_{1}\to M_{2}$ is distance-preserving if for every pair of points x and y in M₁, $d_{2}(f(x),f(y))=d_{1}(x,y).$

This does not constrain the class of structures that are metric spaces, so this definition will not be referred to as an "axiom". See also Axiom § Non-logical axioms.  --Lambiam 05:56, 17 July 2023 (UTC)

@Lambiam: Thank you for clarification. So are there no "additional features of that particular kind of structure" determined in the context of defining independence then? Hildeoc (talk) 12:28, 17 July 2023 (UTC)

Indeed. For example, the silly theorem that an event $X$ is independent of itself iff $\operatorname {P} (X)=0\lor \operatorname {P} (X)=1$ is valid in any probability space.  --Lambiam 17:46, 17 July 2023 (UTC)

@Lambiam: Sorry if I'm being slow-witted here, but how exactly do these exceptions prove that, in general, $P(A\cap B)=P(A)\cdot P(B)$ does not define additional features for independent events – as opposed to $P(A\cup B)=P(A)+P(B)$ defining additional features for disjoint events. I still don't see the conceptual difference making the latter an axiom and the former not. Hildeoc (talk) 18:20, 17 July 2023 (UTC)

PS: What strikes me even more, upon closer examination, is the fact that the additivity axiom is nothing else than a modification of the more general sum rule $P(A\cup B)=P(A)+P(B)-P(A\cap B)$. So why exactly is $P(A\cup B)=P(A)+P(B)$ for disjoint events considered an axiom on its own, in fact? Hildeoc (talk) 19:09, 17 July 2023 (UTC)

The definition of "right angle" does not change the concept of Euclidean space. The properties that make a space an Euclidean space would not be any different if Euclid had forgotten to define right angles. There are spaces in which some of the Euclidean axioms do not hold. For a long time, it was not known whether the parallel postulate was independent of the other axioms, so an intriguing question was, Is it possible to prove the parallel postulate from the other axioms? (This is actually still a meaningful question, but now we know the answer.) Compare this to the question, Is it possible to prove the definition of "right angle" from the Euclidean axioms? The notion of "proving a definition" is meaningless, so the whole question is meaningless.

 

It is possible to have a structure $(\Omega ,{\mathcal {F}},\operatorname {P} )$ in which $\Omega$ is a set, ${\mathcal {F}}$ is a nonempty collection of subsets of $\Omega$ closed under complement, countable unions, and countable intersections, and $\operatorname {P} :{\mathcal {F}}\to [0,1]$ is a function assigning a number in the unit interval to each member of ${\mathcal {F}}$, which does not satisfy the property that $A\cap B=\emptyset$ implies $\operatorname {P} (A\cup B)=\operatorname {P} (A)+\operatorname {P} (B).$ For example, we can choose

• $\Omega =\{0\},$
• ${\mathcal {F}}=\{\emptyset ,\{0\}\},$
• $\operatorname {P} (\emptyset )=\operatorname {P} (\{0\})=1.$

Then $1=\operatorname {P} (\{0\})=$ $\operatorname {P} (\emptyset \cup \{0\})\neq$ $\operatorname {P} (\emptyset )+\operatorname {P} (\{0\})=1+1.$ So the axiom for disjoint events rules out this structure; it would be a probability space but for the last axiom.  --Lambiam 22:19, 17 July 2023 (UTC)

@Lambiam: Sorry, but I guess this is somewhat beyond my ken. I'm not a professional, and I can't follow anymore. Could you maybe somehow break down this argumentation to the main guiding principles for me? Hildeoc (talk) 22:45, 17 July 2023 (UTC)

Removing the parallel postulate from the Euclidean axioms makes a real difference. It makes non-Euclidean geometry possible. Removing the disjointness axiom from the probability axioms also makes a real difference. Without this axiom, both parties in a bet could be sure to win in some model. This model cannot be a model for what we call "probability"; it is no good. Axioms are propositions you can try to test (verify or falsify) in proposed models, to distinguish between good models and no-good models. Definitions are not testable. They are true by definition; if you define a right angle as a quarter turn, you cannot prove that a right angle is not a quarter turn.  --Lambiam 23:29, 17 July 2023 (UTC)

One of Euclid's postulates is: all right angles are equal (i.e. congruent). What precisely do you mean by a Euclidean space without right angles? –jacobolus (t) 23:25, 17 July 2023 (UTC)

I was only considering the possibility of geometry in Euclidean space without introducing the term "right angle". All theorems not using the term (such as the law of cosines) are still theorems. It may be more awkward to prove them (and propositions using the term will then be meaningless, simply because they contain an undefined term), but the properties that make a space Euclidean are not affected.  --Lambiam 23:42, 17 July 2023 (UTC)