Central Limit Theorem

What Is the Central Limit Theorem?

Under appropriate conditions—such as independent or weakly dependent observations and finite variance—the standardized sample mean approaches a normal distribution as sample size grows. Approximation quality depends on the underlying distribution, event rate, tail behavior, and dependence structure; the theorem is not permission to ignore those conditions.

Also Known As

• Data science teams: CLT, Gaussian convergence, asymptotic normality
• Growth teams: "the reason A/B tests work"
• Marketing teams: the normality assumption behind significance tests
• Engineering teams: sampling distribution convergence

How It Works

Imagine running an A/B test where individual conversion outcomes are binary. The mean conversion rate is one draw from a distribution of possible sample means. If the design assumptions hold and the event counts are adequate, a normal approximation may be useful. There is no universal visitor cutoff at which the approximation becomes fully valid.

Best Practices

• Do assess approximation quality from the metric distribution, event rate, sample size, and dependence structure.
• Do validate sample-size requirements before launch rather than hoping CLT will save you.
• Do choose an analytic, exact, simulation, or bootstrap method whose assumptions match the estimand and data.
• Do not assume CLT kicks in instantly for revenue-per-visitor where 95% of users contribute zero.
• Do not confuse "data is normal" with "sample mean is normal" — only the latter is what CLT guarantees.

Common Mistakes

• Calling early wins on tests with 50 conversions per arm, where CLT has not stabilized.
• Applying z-tests to revenue metrics without checking whether the mean's sampling distribution has converged.
• Assuming CLT means you can ignore outliers; large outliers delay convergence.

Industry Context

• SaaS/B2B: Low traffic forces careful use of CLT; demo request tests often need weeks to reach stable sampling distributions.
• Ecommerce/DTC: High traffic makes CLT reliable for conversion tests but revenue-per-visitor still needs outlier handling.
• Lead gen/services: Sparse conversions (1-2%) mean you need very large samples before the CLT approximation is trustworthy.

The Behavioral Science Connection

Kahneman and Tversky's "law of small numbers" describes how humans intuitively expect small samples to reflect population properties. CLT is the mathematical rebuttal: only large samples of means converge to normality. The psychological pull to trust small samples is exactly why experimentation programs need the discipline of fixed sample-size calculations.

Key Takeaway

CLT lets you use normal-distribution math on non-normal data, but only once your sample is large enough for the averages to stabilize.