Bayesian Reasoning: Updating Beliefs About Hypotheses

Priors, Likelihoods, and Posteriors

Bayes' Theorem is a fundamental tool for updating our beliefs (probabilities) about hypotheses in light of new evidence. It elegantly combines:

  • Prior Probability P(H): Our initial belief in a hypothesis H before observing any new data. This can be based on previous knowledge, assumptions, or even a neutral stance (e.g., 50/50).
  • Likelihood P(E|H): The probability of observing the evidence E if our hypothesis H is true.
  • Evidence P(E): The overall probability of observing the evidence, considering all possible hypotheses.

The result is the Posterior Probability P(H|E), our updated belief in hypothesis H after seeing evidence E.

Comparing Competing Hypotheses

Bayes' Theorem is particularly powerful when comparing two or more competing hypotheses. By calculating the posterior probability for each hypothesis given the same evidence, we can determine which hypothesis is more strongly supported by the data. The choice of prior probabilities can influence the outcome, highlighting the importance of stating these assumptions clearly.

Unfair Coin Detection

ADVANCED

You suspect a coin might be unfair. Specifically, you hypothesize that it could be unfair such that the probability of getting Heads, P(H), is only 0.01. You flip this coin 10 times and observe 10 consecutive Heads.

Assuming your prior belief that the coin is this specific type of "unfair" (P(H)=0.01) is 50%, and your prior belief that it's a "fair" coin (P(H)=0.5) is also 50%, what is the probability that the coin is indeed this "unfair" type, given the evidence of 10 Heads?

Hypothesize: How would your conclusion change if your prior belief P(Unfair with P(H)=0.01) was much higher, say 0.99, and P(Fair) was 0.01? Would 10 heads still make you believe the coin is fair?

 

Nerchuko Academy · Free DS Interview Prep