How to Handle Multiple Metrics in A/B Testing
When A/B tests track multiple metrics, statistical complexity increases. Learn frameworks for managing metric conflicts and making sound decisions.
Articles exploring metrics through the lens of behavioral science and experimentation. Practical frameworks for growth leaders who measure in revenue, not vanity metrics.
7 articles
When A/B tests track multiple metrics, statistical complexity increases. Learn frameworks for managing metric conflicts and making sound decisions.
Your primary metric determines whether an A/B test succeeds or fails. Learn how to select metrics that are sensitive, aligned, and actionable.
Visitor-based vs session-based conversion counting, the exact math showing how it changes your reported rate, unique vs all conversions, how to audit your setup, and common bugs that inflate conversions.
The three Optimizely metric types explained for practitioners — when revenue per visitor beats revenue per purchase, the variance problem with revenue metrics, ratio metric gotchas, and practical setup examples.
Why you can only have one primary metric, how to choose it correctly, why revenue per visitor usually beats CVR alone, and how metric selection affects test duration and statistical validity.
"Conversion rate" means completely different things for an ecommerce site vs. SaaS vs. media company. Here's the right metric hierarchy for each revenue model, with Optimizely setup instructions and worked examples.
How to define a smallest effect worth shipping (SEWS) using financial impact, a tight measurement window, and disciplined experiment design.