Back to course

Bayesian Updating with Bayes' Rule

Bayesian updating is Bayes' theorem applied repeatedly. What makes it a process rather than a single calculation is one property: today's posterior becomes tomorrow's prior.

The proportional form
posterior∝likelihood×prior\text{posterior} \propto \text{likelihood} \times \text{prior}

Everything you believed, reweighted by how well each value explains what you have just seen.

The proportionality is deliberate. The denominator P(data)P(\text{data}) is a normalising constant that does not depend on θ\theta, so you can ignore it while working and normalise at the end. That single simplification removes most of the arithmetic.

Sequential updating

Observe data in sequence and update each time:

P(θ)→  d1  P(θ∣d1)→  d2  P(θ∣d1,d2)→  d3  ⋯P(\theta) \xrightarrow{\;d_1\;} P(\theta \mid d_1) \xrightarrow{\;d_2\;} P(\theta \mid d_1, d_2) \xrightarrow{\;d_3\;} \cdots

For conditionally independent observations, this gives exactly the same answer as processing all the data at once, and the order does not matter. Updating on d1d_1 then d2d_2 lands in the same place as d2d_2 then d1d_1.

The rest of this lesson is for subscribers

Unlock every lesson in Fundamentals of Probability and Statistics, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue
NextPrior, Likelihood, Posterior

Keep reading Fundamentals of Probability and Statistics

41 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in every course, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them