Regression Diagnostics
A regression always returns numbers. Diagnostics decide whether those numbers mean anything.
The single most useful habit is to plot residuals against fitted values. A healthy plot is a featureless cloud around zero. Any structure in it is the model telling you what it got wrong.
Work through the four patterns and watch only the lower panel. The fan and the curve are obvious there and nearly invisible in the scatter above, which is the reason the residual plot is the habit rather than the fitted line.
Sort the problems by severity
Not all assumption violations are equally serious, and knowing the ranking saves a great deal of wasted effort.
Biases the coefficients (serious): omitted variables correlated with your predictors, wrong functional form, endogeneity.
Coefficients fine, standard errors wrong (still serious, but different): heteroscedasticity, autocorrelation. Estimates are unbiased, but every t-statistic and p-value is untrustworthy.
The rest of this lesson is for subscribers
Unlock every lesson in Fundamentals of Probability and Statistics, and every other premium course.
Subscribe to continueTest your knowledge
Keep reading Fundamentals of Probability and Statistics
41 lessons in this course, and every other premium course, on one subscription.
- Every lesson in all seven courses, with the worked examples and interactive simulators
- Graded questions on every lesson, with explanations for the wrong answers as well as the right one
- The trainers, timed assessments and brainteaser library that go with them