Minimum Detectable Effect (MDE)

What Is Minimum Detectable Effect (MDE)?

Minimum Detectable Effect is the smallest lift your test can credibly detect at your chosen alpha and power. It is not an aspiration — it is a mathematical floor. If your MDE is 5% and the true effect is 2%, your test will usually return a flat or inconclusive result even when the variant is genuinely better. MDE is the gatekeeper between experiments that matter and experiments that waste a quarter.

Also Known As

• Data science: MDE, minimum detectable lift, detectable delta
• Growth: smallest "win" we can see, sensitivity floor
• Marketing: "what size change will this test actually catch?"
• Engineering: detection threshold, effect sensitivity

How It Works

Suppose your baseline checkout conversion rate is 4%, you run 50,000 visitors per variant, alpha is 0.05, and power is 0.80. Plugging into a standard two-proportion z-test formula, your MDE is roughly a 0.35 percentage-point absolute lift — about 8.75% relative lift. That means a variant that truly lifts conversions by 5% relative will likely fail to reach significance. To detect a 5% relative lift on this baseline you would need roughly 160,000 visitors per variant.

MDE scales inversely with the square root of sample size: doubling traffic does not halve MDE, it divides it by about 1.41.

Best Practices

• Calculate MDE before launch, not after. Use a power calculator at the design stage and publish it in the test doc.
• Compare MDE to realistic business expectations. Use the distribution of effects in your own trustworthy historical data rather than an industry win-rate assumption.
• Budget by traffic, not calendar time. A "two-week test" is meaningless without the sample count.
• Report MDE with every inconclusive result. Flat is not failure — it is information about what you ruled out.
• Use one-sided MDE only when directional hypotheses are truly defensible.

Common Mistakes

• Confusing observed effect with detectable effect. An observed effect below the design MDE is not automatically noise; interpret its interval and test statistic while noting the design had limited planned sensitivity at that size.
• Running low-traffic pages without checking feasibility. Calculate the detectable effect from the actual baseline, variance, traffic, and time available.
• Lowering power to 0.5 or raising alpha to 0.2 to "get answers." Lower power raises false-negative risk; higher alpha raises false-positive risk. Both weaken the decision rule.

Industry Context

In SaaS/B2B, low-volume funnels can make MDE a central design constraint. In ecommerce, traffic and baseline rates vary materially by surface, product, and season. In lead gen, MDE interacts with offline conversion delays. In every case, use the company's own baseline and size for the downstream event that drives the decision.

The Behavioral Science Connection

Stakeholders can treat any number from a dashboard as meaningful. MDE forces an honest conversation about what the design was sensitive enough to detect. An estimate below the design MDE can still be informative, but it will often be less precise than stakeholders expect. Publish MDE and the confidence interval with every readout.

Key Takeaway

MDE is the honesty check on every experiment. Calculate it before you launch, report it with every result, and kill tests that are mathematically incapable of answering the question you care about.