How to Write A/B Test Hypotheses That Don't Suck
"Let's test a bigger CTA" is not a hypothesis. Here's the exact structure for writing A/B test hypotheses that produce useful results whether they win or…
Read Article→
Free Learning · Better Decisions Make Growth Decisions
Make Growth Decisions
You Can Defend
Know what to test, when to trust the result, and what to do next. Practical guides for analysts, growth teams, and founders.
Why Optimizely Data Doesn't Match Google Analytics (And How to Fix It)
Optimizely and GA4 will never show identical numbers — and that's expected.
Read Article→
When to Stop an A/B Test (And Why 95% Confidence Isn't Enough)
Stopping rules for A/B tests: what 95% confidence does and doesn't guarantee, the peeking trap, and how to call a test without wrecking your data.
Read Article→
Segmenting A/B Test Results in Optimizely: Where the Real Insights Hide
The top-line result is often a lie. This guide shows you how to segment Optimizely results correctly, which segments actually matter, and how to avoid the…
Read Article→
How to Read Your Optimizely Results Page (A Complete Walkthrough)
A practitioner's guide to every element on the Optimizely results page — what it means, what to check first, and how to avoid the most common misreads that…
Read Article→
How to Run an A/A Test in Optimizely (And What It Actually Tells You)
Most teams skip A/A tests and only realize the mistake after shipping a 'winner' that quietly reverses.
Read Article→
10 High-Impact A/B Tests You Can Launch in Optimizely This Week
Not all A/B tests are equal. Here are 10 experiments with tight behavioral hypotheses, realistic lift expectations, and the exact failure modes to watch out…
Read Article→
A/B vs Multivariate vs Multi-Page Testing: Which Should You Run?
The wrong test type is one of the most common ways CRO programs waste months.
Read Article→
Why You Can Never Change a Running Experiment (And What To Do Instead)
Someone changed your live A/B test. Maybe it was you. Here's exactly what that broke, why the data is compromised, and the step-by-step rescue workflow to…
Read Article→
How Long Should You Run an A/B Test? (The Real Answer)
Seven years running 100+ experiments taught me that test duration is the most violated rule in CRO.
Read Article→
Minimum Detectable Effect (MDE): The Most Important Number You're Not Setting Correctly
MDE isn't a calculator input — it's the foundation of your entire experiment design.
Read Article→
Bayesian vs Frequentist Testing in Optimizely: Which Should You Choose?
Optimizely now offers three statistical engines: Sequential (Stats Engine), Frequentist Fixed Horizon, and Bayesian.
Read Article→
Statistical Significance in Optimizely: What It Really Means
How Optimizely calculates statistical significance, what 95% actually tells you, and the common misreadings that cost teams real money.
Read Article→
Why Your Optimizely Results Keep Changing (And When to Worry)
Tuesday your experiment shows 94% confidence. Friday it's 71%. Nothing changed — so what's happening?
Read Article→
False Discovery Rate in Optimizely: Why Running Many Tests Simultaneously Is Riskier Than You Think
Running 20 tests at 95% confidence means you expect at least one false positive by chance.
Read Article→
Why Your Optimizely Test Isn't Statistically Significant
Five reasons Optimizely experiments stall below statistical significance — sample size, MDE, traffic allocation — and the fix for each one.
Read Article→
Homepage Testing Is Overrated: Evidence from 16 Homepage Experiments
16 homepage A/B tests exposed a 69% inconclusive rate — worse than any other page type. Data shows downstream pages win at 2x the rate.
Read Article→
Why Your Pricing Page A/B Test Failed: Lessons from 13 Pricing Experiments
We ran 13 pricing page A/B tests with a 15% win rate. Here are the counterintuitive lessons about why pricing psychology fails in practice.
Read Article→
Why 61% of A/B Tests Are Inconclusive — And Why That’s Exactly Right
Most A/B tests don't produce winners. Our data from 97 experiments reveals why a 61% inconclusive rate signals a rigorous program, not a broken one.
Read Article→
The Social Proof Experiment That Failed: Why Evidence Beats Intuition
We ran 3 social proof A/B tests and got 0 winners. Here is why the most recommended conversion tactic failed and what actually works instead.
Read Article→
Mobile vs. Desktop: 13 Experiments That Challenge Responsive Design Assumptions
Analysis of 13 mobile A/B tests reveals a 38% win rate, beating desktop. Learn why device-specific testing matters more than responsive design alone.
Read Article→
Behavioral Science Claims vs. Experiment Reality: What Actually Moves the Needle
97 real A/B experiments tested behavioral science principles in the field.
Read Article→
The Hidden Cost of Running Zero Experiments: A Business Economics Analysis
Most companies quantify testing costs but never calculate what NOT testing costs. The Experiment P&L framework reveals the true economics.
Read Article→
Product Comparison Pages: 19 Experiments Reveal What Actually Converts
19 A/B tests on product comparison pages reveal layout structure beats content additions, attribute curation drives decisions, and interactive builders…
Read Article→
Better Decisions
Make Growth Decisions
You Can Defend
Know what to test, when to trust the result, and what to do next. Practical decision guides for analysts, growth teams, and founders. Free. Weekly.
Opens Substack to confirm your subscription · Free · Unsubscribe anytime