Prior Distribution
In Bayesian statistics, the probability distribution representing existing beliefs or knowledge about a parameter before observing new data, which is combined with evidence to form the posterior.
What Is a Prior Distribution?
A prior encodes what you believe about a parameter before seeing the data. In A/B testing, priors typically describe the plausible range of lift effects. Bayesian inference multiplies the prior by the likelihood of the observed data to produce a posterior, which is your updated belief.
Also Known As
- Data science teams: prior, prior belief, Bayesian prior
- Growth teams: "what we think the lift could be"
- Marketing teams: baseline expectation
- Engineering teams: prior, regularization (in ML contexts)
How It Works
Imagine a Bayesian A/B test with 10,000 visitors per variant. Your historical tests suggest lifts are rarely above 5% and typically between -2% and +3%. You encode this as a Normal(0, 2%) prior — centered at zero, with 95% of mass between -4% and +4%. If Variant B observes a 4% lift, the posterior pulls toward ~2.5% because the prior was skeptical. With a flat prior, the same data would yield a posterior centered at ~4%. The prior tempered your conclusion because extreme lifts are implausible historically.
Best Practices
- Do use weakly-informative priors as a default; they temper noise without dominating data.
- Do base informative priors on documented historical results, not gut feeling.
- Do run sensitivity analyses showing how different priors change the conclusion.
- Do not use priors to manufacture a desired result; that is p-hacking with extra steps.
- Do not assume flat priors are "objective"; they encode a specific belief about the parameter space.
Common Mistakes
- Choosing an overly informative prior that drowns out the data.
- Failing to document the prior; future readers cannot evaluate the conclusion.
- Treating priors as one-time choices rather than iterating as the program learns.
Industry Context
- SaaS/B2B: Informative priors help small-sample tests reach usable conclusions faster.
- Ecommerce/DTC: Rich historical data makes data-driven priors especially powerful.
- Lead gen/services: Sparse conversions benefit enormously from prior shrinkage.
The Behavioral Science Connection
Priors formalize the behavioral insight Kahneman highlighted in his work on base rates: without anchoring to known distributions of outcomes, we over-react to individual cases. A prior is statistical anchoring done on purpose — productive anchoring rather than the cognitive kind.
Key Takeaway
A prior is a statistical commitment about the plausible range of outcomes; choose it deliberately and document it.