Articles exploring methodology through the lens of behavioral science and experimentation. Practical frameworks for growth leaders who measure in revenue, not vanity metrics.
5 articles
A practical guide for new testing teams, CRO managers, and analysts. The five most common statistical mistakes in DTC A/B testing, why each one happens, and exactly what to do instead — with quick rules of thumb, definitions, and Monday-morning action items.
New CRO analysts learn the field through case studies — but case studies are curated by construction, and reading them as evidence produces an inflated model of what good A/B testing looks like. This guide teaches you how to extract the useful parts without being misled by the curation, with a 5-step reading protocol you can apply today.
A test that changes two systems at once produces an answer you cannot use. The fix isn't "test smaller" — it's a recovery sequence that decomposes the bundle into actionable iterations. Here is the discipline that turns a failed redesign into a roadmap.
Standard 50/50 A/B testing fails on pages with high baseline conversion or thin traffic — the math says the test cannot detect a meaningful effect in any reasonable runtime. Across 200+ tests in a real portfolio, only 2% reached traditional stat sig. The other 98% needed a different methodology to ship defensibly.
An anonymized analysis of 200+ A/B tests run across an enterprise CRO program. The aggregate data contradicts most published CRO advice — only 2% of tests reach traditional statistical significance, single-device tests outperform "combined" tests by 3×, and 89% of testing budget targets acquisition while retention is starved.