Credible Interval
The Bayesian counterpart to a confidence interval, representing a range within which the true parameter value falls with a specified probability, such as 95%, given the observed data and prior.
What Is a Credible Interval?
A 95% credible interval is a range that contains the true parameter with 95% probability given your data and prior. Unlike a frequentist confidence interval, which has a convoluted definition about hypothetical repeated experiments, a credible interval means exactly what people intuitively assume a confidence interval means.
Also Known As
- Data science teams: Bayesian interval, posterior interval, credible region
- Growth teams: "the 95% lift range"
- Marketing teams: probability-based range estimate
- Engineering teams: HDI (highest density interval), equal-tailed interval
How It Works
Imagine a Bayesian A/B test with 10,000 visitors per variant. The posterior on lift is approximately Normal(1.3%, 0.28%). The 95% credible interval is roughly [0.75%, 1.85%]. You can honestly tell your PM: "There is a 95% probability the true lift is between 0.75% and 1.85%." Compare this to the frequentist confidence interval which technically only makes sense in the context of repeated sampling — a distinction stakeholders never actually absorb.
Best Practices
- Do report credible intervals alongside posterior means for full uncertainty context.
- Do use highest density intervals (HDI) when posteriors are skewed.
- Do anchor decisions on whether the interval excludes zero or a business threshold.
- Do not compare credible intervals across tests with different priors without acknowledging the mismatch.
- Do not over-interpret narrow intervals; narrow is not the same as meaningful.
Common Mistakes
- Reporting credible intervals without specifying the credibility level (90%, 95%, 99%).
- Treating credible intervals and confidence intervals as identical; they can differ substantially.
- Forgetting that credible intervals depend on the prior; flat priors are a specific choice.
Industry Context
- SaaS/B2B: Credible intervals help explain uncertainty to non-technical stakeholders.
- Ecommerce/DTC: Decision rules like "ship if lower bound > 0.5%" use credible intervals directly.
- Lead gen/services: Wide credible intervals honestly communicate the cost of small samples.
The Behavioral Science Connection
Credible intervals combat the "precision illusion" Kahneman documents — the human tendency to treat any number as exact. By explicitly showing a range, they force stakeholders to reason about uncertainty rather than anchoring on a single point estimate.
Key Takeaway
Credible intervals say what people wish confidence intervals said; use them to communicate uncertainty honestly.