# Against the odds: a caution to practical Bayesians

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Lots of people I know like to use Bayes' Theorem in their daily life to estimate the odds that various statements are true. See http://betterexplained.com/articles/understanding-bayes-theorem-with-ratios/ for a quick tutorial. This post is about a pitfall to watch out for once you're feeling pretty good about applying Bayes' Theorem.

Upshot: if you do multiple Bayesian updates on the odds of a proposition (i.e. P(X):P(¬X)), then you're probably making errors unless X and ¬X are pretty simple hypotheses.

What went wrong? First of all, that 1:9 likelihood ratio is bogus. If we condition on the thief not being Alice, the probablity of a witness saying some specific person other than Alice (say Bob) did it is really $$P(witness_B|¬A) = 0.8×P(B|¬A) + 0.1×P(C|¬A) = 0.8×0.5 + 0.1×0.5 = 0.45,$$
so the likelihood ratio should have been 10:45 = 2:9 instead of 1:9. But using this corrected ratio still gives us the wrong answer for a deeper reason: if you condition on ¬Alice, then the witness testimonies are not logically independent! Once I hear one testimony against Bob, then when compute $P(witness_B|¬A)$, $P(B|¬A)$ and $P(C|¬A)$ aren't 0.5 anymore. Note that this is consistent with the assumption that the witness testimonies are causally independent.