Back to course

p-values and Statistical Decisions

A p-value is the probability of observing data at least as extreme as what you saw, assuming the null hypothesis is true.

That conditional clause is the whole definition, and dropping it produces every common error.

The p-value
p=P(data this extreme or more∣H0 true)p = P(\text{data this extreme or more} \mid H_0 \text{ true})

The probability of data this extreme IF the null is true. It is not the probability that the null is true.

Small pp means the data would be surprising if the null held. That is all it means.

Three things it is not

Not the probability the null is true. P(data∣H0)P(\text{data} \mid H_0) is not P(H0∣data)P(H_0 \mid \text{data}), and converting between them requires a prior, which is Bayes' theorem. A p=0.04p = 0.04 result testing an implausible strategy is far more likely to be a fluke than the same p-value testing a well-motivated one.

The rest of this lesson is for subscribers

Unlock every lesson in Fundamentals of Probability and Statistics, and every other premium course.

Subscribe to continue

Test your knowledge

Questions are only available to subscribers.
Continue
NextType I and Type II Errors

Keep reading Fundamentals of Probability and Statistics

41 lessons in this course, and every other premium course, on one subscription.

Subscribe now
  • Every lesson in every course, with the worked examples and interactive simulators
  • Graded questions on every lesson, with explanations for the wrong answers as well as the right one
  • The trainers, timed assessments and brainteaser library that go with them