Back to course

Likelihood Ratio Tests

A richer model always fits the data at least as well. The question is whether the improvement exceeds what extra parameters would buy by chance alone, and the likelihood ratio test answers exactly that.

The test

For nested models, meaning the simpler one is the richer one with parameters constrained:

Wilks' theorem
−2log⁡Λ=−2(ℓ0−ℓ1)∼χk2-2\log\Lambda = -2\left(\ell_0 - \ell_1\right) \sim \chi^2_k

Twice the log-likelihood gap is chi-squared, which is what turns comparing two models into a test.

where ℓ0\ell_0 and ℓ1\ell_1 are maximised log-likelihoods and kk is the difference in parameter count.

Large values mean the extra parameters bought more fit than chance would supply. Compare against a chi-squared critical value with kk degrees of freedom.

Key takeaway

Nesting is required. The test compares a model against a restricted version of itself, and it says nothing about two models that are not related that way.

The rest of this lesson is for subscribers

Unlock every lesson in Advanced Topics in Probability and Statistics, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue
NextShrinkage Estimators and Empirical Bayes

Keep reading Advanced Topics in Probability and Statistics

35 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in every course, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them