The Traffic Question Everyone Gets Wrong
The most common question in experimentation is "Do I have enough traffic to A/B test?" The most common answer — a fixed number like ten thousand visitors per month — is wrong.
Traffic requirements are not a fixed threshold. They depend on what you are testing, how big of an effect you expect, and what your current conversion rate looks like. A site with moderate traffic can absolutely run rigorous tests, and a site with high traffic can still design tests that fail to reach significance.
Here is how to think about traffic requirements correctly.
Why Traffic Matters: The Statistical Foundation
A/B testing is a statistical comparison between two groups. To confidently say one version is better than the other, you need enough observations to distinguish a real difference from random noise.
Think of it like listening for a specific sound in a noisy room. If the sound is loud (a large effect), you can hear it quickly even in moderate noise. If the sound is quiet (a small effect), you need a quieter room (more data) to detect it.
In A/B testing terms:
- The "sound" is the effect size — how much better (or worse) the variant is compared to the control
- The "noise" is random variation — the natural fluctuation in user behavior
- The "room" is your sample size — more visitors means less noise relative to signal
The Four Factors That Determine Required Traffic
1. Baseline Conversion Rate
Your current conversion rate sets the starting point. Higher baselines are easier to work with because you observe more conversion events per visitor.
A page where a moderate fraction of visitors convert generates enough conversion events to detect changes relatively quickly. A page where only a tiny fraction convert needs many more visitors to accumulate enough conversion events for a valid comparison.
2. Minimum Detectable Effect (MDE)
This is the smallest improvement you want your test to be able to detect. It is expressed as a relative change from your baseline.
Smaller MDEs require exponentially more traffic. Detecting a small relative improvement requires roughly four times more traffic than detecting a moderate one. This is not a linear relationship — halving your MDE roughly quadruples your sample size.
The MDE should be a business decision. Ask: "What is the smallest improvement that would be worth implementing this change?" If a small relative lift is not worth the development cost, set your MDE higher and save traffic.
3. Statistical Significance Level
This is the probability of a false positive — declaring a winner when there is no real difference. The standard is five percent (ninety-five percent confidence).
Lower significance levels (higher confidence) require more traffic. Most teams should stick with the ninety-five percent standard.
4. Statistical Power
Power is the probability of detecting a real effect when one exists. The standard is eighty percent, meaning if there is a true difference of your MDE or larger, you have an eighty percent chance of detecting it.
Higher power requires more traffic. Some teams use ninety percent power for critical tests, which increases the sample size by roughly a third.
Calculating Your Required Sample Size
The formula connects all four factors. Use an online sample size calculator — there is no need to compute this manually.
Input your baseline conversion rate, desired MDE, significance level, and power. The calculator outputs the number of visitors needed per variant. Double that for a standard two-variant test.
Example scenarios to build intuition:
- A page with a moderate baseline rate testing for a moderate relative improvement might need several thousand visitors per variant
- The same page testing for a small relative improvement might need tens of thousands per variant
- A page with a low baseline rate testing for a moderate improvement might need tens of thousands per variant
The numbers scale predictably. Lower baselines and smaller MDEs always mean more required traffic.
What If You Do Not Have Enough Traffic?
Low traffic does not mean you cannot test. It means you need to adapt your approach.
Test bigger changes
Larger changes produce larger effects, which need fewer observations to detect. Instead of testing a minor copy tweak, test a fundamentally different value proposition. Instead of swapping a button color, redesign the entire call-to-action section.
The tradeoff: bigger changes are harder to interpret because multiple elements changed. But a test that produces a clear result on a big change is more useful than a test that runs for months on a small change and never reaches significance.
Test higher-stakes pages
Concentrate testing on your highest-traffic pages where each improvement has the greatest business impact. A small site might only have enough traffic to test its homepage or primary landing page, and that is fine — those are usually where the biggest opportunities live anyway.
Use composite metrics
Instead of testing a single low-frequency action (like purchases), test a higher-frequency proxy metric that correlates with it (like add-to-cart clicks). Proxy metrics generate more events per visitor, reducing the required sample size.
Extend the test duration
If you cannot increase traffic to the page, you can extend the test runtime. A page with low daily traffic that runs for six weeks collects the same total sample as a high-traffic page that runs for one week. The risk is that longer tests are more vulnerable to external changes.
Reduce the number of variants
Every variant you add divides your traffic further. If traffic is limited, stick to a simple A/B test (two variants) rather than an A/B/C/D test.
Traffic Requirements for Different Test Types
Different testing approaches have different traffic needs:
Standard A/B tests (two variants) have the lowest requirements. All traffic is split between two versions, maximizing the sample per variant.
A/B/n tests (multiple variants) divide traffic among more groups. Three variants means each gets a third of your traffic, so you need roughly three times the total traffic of a two-variant test.
Multivariate tests have the highest traffic requirements because every combination of variables needs its own sample. The traffic required grows multiplicatively with each variable added.
The Opportunity Cost of Not Testing
Teams sometimes conclude they do not have enough traffic to test and stop there. This is a mistake.
Every change you ship without testing is a gamble. Some changes improve performance. Some hurt it. Without testing, you do not know which is which. The cost of shipping losers is invisible but real — it compounds over time as bad decisions stack on top of each other.
Even if you can only run one well-powered test per month, that is twelve validated decisions per year. Twelve decisions where you know the impact instead of guessing. That is enormously valuable.
When You Truly Cannot Test
There are situations where A/B testing is genuinely impractical:
- Very low traffic pages (fewer than a handful of visitors per day) will never accumulate enough data for a standard A/B test. Consider qualitative methods instead: user testing, surveys, session recordings.
- Rare conversion events (enterprise sales with very few closings per month) cannot be measured with standard A/B testing sample sizes. Test earlier-funnel metrics or use qualitative evaluation.
- One-time events (a product launch, a single campaign) do not provide enough repeated observations for statistical comparison.
In these cases, use other research methods (user testing, expert review, analytics analysis) to make informed decisions. A/B testing is powerful but it is not the only way to reduce decision uncertainty.
Building Traffic Over Time
If your current traffic is insufficient for the tests you want to run, build traffic while testing what you can.
Content marketing, SEO, and paid acquisition all increase the traffic pool available for testing. As traffic grows, your testing program can become more ambitious — smaller MDEs, more simultaneous tests, multivariate approaches.
Treat your traffic as a finite testing resource and allocate it strategically. Run your most important tests on your highest-traffic pages first. Save low-traffic pages for after you have optimized the high-impact areas.
FAQ
Is there a minimum traffic threshold for A/B testing?
There is no universal minimum. The required traffic depends on your conversion rate and MDE. A page with a few hundred daily visitors can run meaningful tests if you test big changes with moderate-to-large expected effects. The key is running the sample size calculation before committing to a test.
Can I combine traffic from multiple pages into one test?
Only if the pages serve the same purpose and the change applies identically across them. Combining traffic from your homepage and a product page into one test mixes different user intents and produces unreliable results. Combining traffic from two identical landing pages that differ only in traffic source is acceptable.
How does mobile vs desktop traffic affect requirements?
Mobile and desktop users often behave differently. If you run a test across both, the different behavior patterns add noise. For cleaner results, either segment by device (which divides your traffic) or ensure your variant works equally well on both.
What about testing in B2B with low-volume, high-value conversions?
B2B companies with long sales cycles and few conversions per month should test upstream metrics: demo request rate, content downloads, pricing page engagement. These higher-frequency events are testable even with modest traffic.
How do I explain traffic requirements to stakeholders?
Use a simple analogy: a test is like a clinical trial. Running it with too few participants produces unreliable results that could lead to bad decisions. The sample size is the minimum needed to trust the result. Cutting it short is like stopping a medical trial early — you might get lucky, but you might also make a harmful mistake.