Experiment Velocity
The rate at which an organization runs experiments — typically measured in tests per month or quarter — a leading indicator of experimentation program maturity.
Experiment velocity is the most reliable predictor of experimentation ROI. Organizations that run more experiments learn faster, compound more insights, and generate more revenue per test. The correlation between velocity and revenue impact is well-documented across industry benchmarks.
Velocity Benchmarks
Early-stage programs: 2-5 tests/month. Maturing programs: 10-20 tests/month. World-class programs (Booking.com, Netflix, Amazon): 100+ tests running simultaneously. The goal isn't to match world-class velocity immediately — it's to increase velocity quarter over quarter.
Velocity vs. Quality
The common objection: "We should run fewer, better tests." This sounds wise but is empirically wrong. Higher velocity doesn't mean lower quality — it means faster learning. The teams with the highest velocity also have the highest win rates, because rapid iteration produces better hypotheses over time.
How to Increase Velocity
The bottleneck is rarely ideas — it's implementation. To increase velocity: standardize test implementation (use a testing platform, not custom code), reduce approval cycles (test review should take days, not weeks), and create a shared hypothesis backlog that any team member can pull from.
What Hiring Managers Look For
When building experimentation teams, I look for candidates who understand velocity as a system metric, not just a count. Can they identify bottlenecks? Design processes that reduce test implementation time? Build a culture where running an inconclusive test is acceptable because the learning has value?