Back to course

Conjugate Priors in Exponential Families

A conjugate prior produces a posterior in the same family as the prior. Updating becomes parameter arithmetic rather than integration.

Beta-binomial conjugacy
θ∼Beta(α,β),X∣θ∼Binomial(n,θ)  ⟹  θ∣X∼Beta(α+x,  β+n−x)\theta \sim \text{Beta}(\alpha,\beta), \quad X \mid \theta \sim \text{Binomial}(n,\theta) \implies \theta \mid X \sim \text{Beta}(\alpha + x,\; \beta + n - x)

The posterior stays in the prior's family, so updating is arithmetic on parameters rather than an integral.

Observe xx successes and n−xn-x failures; add them to the parameters. That is the entire update.

The pairs to know

Likelihood Conjugate prior Update
Binomial Beta Add successes and failures
Poisson Gamma Add counts and exposure
Normal (known variance) Normal Precision-weighted average
Normal (unknown variance) Normal-inverse-gamma Both parameters update
Exponential Gamma Add observations and total time

These all arise because the likelihoods belong to the exponential family, whose members share a structure that guarantees a conjugate prior exists.

The rest of this lesson is for subscribers

Unlock every lesson in Advanced Topics in Probability and Statistics, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue
NextBayesian Decision Theory

Keep reading Advanced Topics in Probability and Statistics

35 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in every course, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them