The experimentation playbooks are written by companies that can ship a test in an afternoon. Inside a Fortune 150 enterprise, that same test can take six weeks to go live — not because the analysis is hard, but because it has to clear legal, brand, and three levels of sign-off first. The velocity advice doesn't transfer, and pretending it does is why enterprise programs stall.
TL;DR
- Most CRO advice assumes startup velocity that enterprises structurally cannot match. The bottleneck at scale isn't test design or statistics — it's the approval chain a test must clear before it can go live.
- The root error is a category mistake: enterprises treat an A/B test like an irreversible, one-way-door decision when it is the single most reversible decision in the building — a two-way door with an undo button built in.
- The cost is invisible on any dashboard. It shows up as the tests never proposed because the process isn't worth it, and the market windows that close while a test waits in a review queue.
- The reframe that unlocks velocity is governance proportional to reversibility: heavyweight review for the few genuinely irreversible changes, a fast lane for the many reversible ones.
| Decision type | Reversibility | Right process | Where A/B tests belong |
|---|---|---|---|
| One-way door | Hard/impossible to undo | Deliberate, senior sign-off | ❌ Not here |
| Two-way door | Easy to undo | Fast, delegated | ✅ Here |
| A controlled experiment | Undo built in (just stop it) | Should be fastest lane | ✅✅ Most reversible thing you run |
The enterprise instinct is to route every test through the one-way-door process. That instinct is precisely backwards, and it's the reason velocity dies.
The advice gap nobody names
Read the canonical experimentation writing and you'll find velocity treated as a near-pure function of will and tooling: run more tests, fail fast, let the data decide. Kohavi and Thomke's research at Bing and Microsoft documents organizations running tens of thousands of experiments a year, where the culture and infrastructure make each test cheap (Kohavi & Thomke, *The Surprising Power of Online Experiments*, HBR). That's real, and it's aspirational, and it quietly assumes a world most enterprise practitioners don't live in.
In a large regulated company, a test isn't gated by engineering effort. It's gated by who has to say yes before it ships. A copy change touches brand guidelines, so brand reviews it. A pricing-adjacent test touches legal exposure, so legal reviews it. A customer-facing change touches three product lines, so three owners weigh in. Each reviewer is individually reasonable. Collectively they form a serial approval chain where the test's cycle time is the sum of every queue it waits in — and by the time it clears, the seasonal window or competitive moment that motivated it has often passed. The literature optimizes the part of the process that's already fast and ignores the part that's actually slow.
The category error: treating reversible decisions as irreversible
The most useful frame for what's going wrong comes from Jeff Bezos's 2015 shareholder letter, which splits decisions into two types. Type 1 decisions are one-way doors — hard or impossible to reverse, so they deserve slow, deliberate, senior-level scrutiny. Type 2 decisions are two-way doors — easily reversible, so they should be made quickly by the people closest to the work. Bezos's warning is that as organizations grow, they tend to apply the heavyweight Type 1 process to Type 2 decisions, and the result is "slowness, unthoughtful risk aversion, failure to experiment sufficiently, and consequently diminished invention" (Amazon 2015 Letter to Shareholders).
That failure mode is the entire story of enterprise experimentation velocity. A controlled experiment is the most reversible decision an organization can make — more reversible than a Type 2 decision, because it doesn't even fully commit. It exposes a change to a fraction of traffic, measures the effect, and can be switched off instantly if anything looks wrong. The undo isn't just possible; it's built into the mechanism. And yet enterprises routinely subject that maximally-reversible action to their most conservative, one-way-door governance. It's a category error: applying irreversible-decision caution to the one thing specifically designed to be reversible.
An A/B test is a two-way door that also has a fire alarm. Governing it like a one-way door isn't prudence — it's a misread of what the test actually is.
The cost you can't see on a dashboard
The damage from an over-heavy approval chain is real but structurally invisible, which is why it persists. It shows up in two places no report captures:
The tests never proposed. When getting a test live takes six weeks of sign-off, practitioners stop proposing the marginal ones. Why fight the process for a test with a modest expected lift? So the program's idea funnel narrows to only the tests worth a bureaucratic battle, and the long tail of cheap, reversible experiments — exactly the ones that compound into a real velocity advantage — never get run, and the program never builds the experiment velocity that comes from running many small tests. The dashboard shows the tests you ran; it can't show the ones the process discouraged.
The windows that closed. A test motivated by a seasonal moment, a competitive move, or a campaign launch has a shelf life. If the approval chain outlasts the window, the test ships into an irrelevant context or doesn't ship at all. The opportunity cost is total and unrecorded.
This is the enterprise version of the trade-off in when testing costs more than it teaches — except the cost isn't the test's own effort, it's the approval overhead wrapped around it. When that overhead is fixed and high, it changes which tests are even worth running, and it systematically kills the small reversible ones that should be a program's bread and butter.
The reframe: govern proportional to reversibility
The fix is not to abolish review — some changes genuinely carry legal or brand risk that warrants a one-way-door process. The fix is to stop applying that process to everything. Sort experiments by reversibility and irreversible impact, and route them accordingly.
When I led experimentation inside a Fortune 150 enterprise and, before that, a Fortune 500 bank, the bottleneck was never the analysis — it was the approval chain wrapped around every test. A practical version of what actually unlocked velocity inside those large, cautious organizations:
- Define a fast lane by risk, not by politics. Reversible, low-blast-radius tests — copy, layout, non-pricing UX on a bounded traffic slice — get a lightweight, pre-approved review with delegated sign-off. The test is reversible; govern it like it is.
- Reserve heavyweight review for genuine one-way doors. Anything touching pricing commitments, legal representations, or irreversible customer-facing promises keeps the full chain. That's where the caution belongs.
- Pre-clear the guardrails once, not per-test. Get legal and brand to agree up front on the boundaries of the fast lane — what copy is safe, what traffic caps apply, what auto-shutoff conditions trigger. Then individual tests inside those boundaries don't re-litigate the whole thing. This is the single highest-impact move: you're converting a per-test approval into a one-time policy approval.
- Sell the reframe in the stakeholders' own language. Legal and brand aren't obstacles; they're managing real risk. The unlock is showing them that a controlled experiment reduces their risk versus a blanket rollout — it limits exposure to a fraction of traffic and can be killed instantly. Framed that way, the fast lane isn't a bypass of their concern; it's a better tool for it.
The teams that get velocity inside enterprises aren't the ones who fought governance — they're the ones who re-educated it about reversibility, so the process caution landed where it belonged. This is a specific application of a broader truth about experimentation programs: the job is as much internal-consulting and stakeholder alignment as it is running tests, and the reframe from one-way to two-way door is the most important piece of that alignment work.
FAQ
Isn't heavy review just the price of operating in a regulated industry?
Some of it is — genuinely irreversible, legally exposed changes should get deliberate review, and that's non-negotiable. The error isn't having a heavyweight process; it's applying that process indiscriminately to reversible tests that don't warrant it. Regulated industries need one-way-door governance for one-way-door decisions. They lose nothing legitimate by fast-tracking the reversible experiments that make up most of a program.
How do I convince legal and brand to accept a fast lane?
Lead with risk reduction, not speed. A controlled experiment limits a change's exposure to a fraction of users and can be switched off instantly — that is strictly less risky than the blanket rollouts these teams already approve. Then pre-negotiate the fast lane's boundaries once, so they're approving a bounded policy rather than re-reviewing every test. You're not asking them to lower their standards; you're giving them a more precise instrument for the same standards.
Doesn't a fast lane invite sloppy or risky tests?
Only if the fast lane has no guardrails. Define it by traffic caps, reversibility, and auto-shutoff conditions, and the fast lane is safer than the slow one, because it forces explicit limits that ad hoc rollouts never have. The tests in it are reversible by construction. Sloppiness is a function of missing guardrails, not of speed — and the guardrails are what you pre-clear once.
Why not just run every test regardless of the process?
Because inside a real enterprise, shipping around legal and brand isn't velocity — it's a career-ending incident waiting to happen, and it burns the stakeholder trust the program runs on. The durable path is to change the process so the reversible tests move fast with legal and brand's blessing, not to route around the people managing real risk. Speed bought by ignoring governance gets paid back with interest.
Bottom line
Enterprise experimentation doesn't stall because the analysis is hard; it stalls because a serial approval chain treats every test as an irreversible decision. That's a category error: a controlled experiment is the most reversible action an organization can take — a two-way door with an undo built in — and subjecting it to one-way-door governance produces exactly the "diminished invention" Bezos warned about. The cost is invisible: the tests never proposed and the windows that closed while a test waited in a queue. The fix is governance proportional to reversibility — a pre-cleared fast lane for reversible tests, heavyweight review reserved for genuine one-way doors, and the whole reframe sold to legal and brand as risk reduction. Get that right and enterprise scale stops being a velocity tax.
Helping large organizations turn experimentation into a fast, repeatable process — governance and all — is exactly why I built GrowthLayer. For more on the organizational reality of running experimentation at enterprise scale, subscribe to Lean Experiments.