Theoretical vs Empirical Probability

There are two main ways to understand probability.

Theoretical probability is based on ideal models where outcomes are equally likely. For example, the probability of flipping a fair coin and getting heads is \( \frac{1}{2} \). This approach assumes perfect conditions and no external influences.

Empirical probability, on the other hand, is grounded in observed data. If you flip a coin 10 times and observe 6 heads, your empirical probability for heads is \( \frac{6}{10} = 0.6 \). While this may differ from the theoretical value, empirical results tend to converge to theoretical probabilities as the number of trials increases.

This convergence is explained by the Law of Large Numbers, which states that as a process is repeated many times, the average of the observed outcomes approaches the expected value:

\[ \lim_{n \to \infty} \frac{1}{n} \sum_{i=1}^{n} X_i = \mu\]