Atticus Li has led enterprise analytics and experimentation programs where finance, marketing, and product teams brought different evidence to the same budget decision. This guide compares those evidence classes without turning any modeled impact into guaranteed return.
In budget meetings, brand, performance, and experimentation teams often bring different kinds of evidence. Brand may bring lift studies, performance may bring attributed ROAS, and experimentation may bring randomized test results plus an annualized business-impact model. Those numbers are not interchangeable, and none should be presented without its assumptions.
The Attribution Problem Nobody Talks About Honestly
Every marketing function has an attribution problem. The question is how big the asterisk is.
Brand marketing can be measured with surveys, brand tracking, geo experiments, matched markets, media holdouts, and marketing-mix models. Some designs are descriptive; others support stronger incremental conclusions. The evidence label should say which design was used and how the business outcome was connected to it.
Performance marketing is better positioned. They have attribution models. They can show that a user clicked an ad and then purchased. The ROAS calculation is relatively clean for last-click attribution.
But performance marketing has its own asterisk. Last-click attribution gives all credit to the final touchpoint, even if the user was already going to buy. Multi-touch attribution distributes credit according to a model. A reported ROAS may include users who would have converted organically, so the decision-grade question is incremental return. Holdouts, conversion-lift studies, and geo experiments can address that question when the design is valid.
There's also the platform self-reporting problem. Google tells you how well Google ads performed. Meta tells you how well Meta ads performed. Both platforms have incentives to overcount their own contribution. Cross-platform deduplication is a nightmare that most teams solve with heuristics and hope.
Experimentation can isolate the effect of a treatment when randomization, instrumentation, exposure, and analysis are valid. The direct evidence is the difference observed during the test for that population, period, and metric.
The annual dollar figure is a second layer. It usually extrapolates the test-window effect using eligible traffic, adoption, revenue or contribution per conversion, and an assumption about persistence. That model can be useful, but it is not assumption-free and should not be mislabeled as booked revenue.
Why This Matters in Budget Conversations
CFOs should be skeptical of marketing claims. Experimentation's advantage is not that every program dollar is automatically causal; it is that a valid randomized test can provide a strong local estimate before the team models broader financial impact.
A useful budget story combines the portfolio readout with the underlying test designs, annualization assumptions, recognized post-launch outcomes, and any holdout evidence. A historical company result should never become a forecast for another company.
How to Present the Advantage Without Making Enemies
Now, there's a political reality to navigate here. If you walk into the budget meeting and explicitly position experimentation as more rigorous than brand marketing and performance marketing, you'll win the argument and lose the war. Those teams will stop collaborating with you, and collaboration is essential — many of your best experiments optimize the traffic those teams generate.
Here's how I frame it instead.
Lead with the methodology, not the comparison. I don't say "our measurement is better than yours." I say, "A valid randomized experiment helps us isolate the test-window effect. We show the observed result separately from the assumptions used to estimate longer-term business impact."
Acknowledge the ecosystem. Experiments do not happen in a vacuum. Acquisition teams supply traffic, brand work affects how users respond, product teams implement changes, and finance supplies the unit-economics inputs. Credit the system rather than claiming the experiment program created the whole outcome alone.
Frame it as portfolio diversification. "A strong marketing investment portfolio includes brand building for long-term awareness, performance marketing for scalable acquisition, and experimentation for conversion optimization and causal measurement. Each plays a different role, and together they compound."
This framing makes the evidence comparable while preserving the relationships needed to run the work.
The Specific ROI Calculation
Here is the structure I use to present experimentation economics to finance.
First, report the observed test-window effect with its uncertainty. Then show a modeled annual-impact range:
Eligible traffic × baseline conversion × observed relative effect × contribution per conversion × adoption × persistence adjustment
Keep that modeled range separate from revenue or contribution actually recognized after launch. Then compare the appropriately labeled impact with the fully loaded program cost, including headcount, tools, and implementation resources.
The output is a decision model, not a guaranteed return multiple. Finance should be able to inspect every input and replace assumptions with realized post-launch evidence as it becomes available.
The CFO Talk Track
When I present to finance, I use this structure.
Start with the evidence label. State whether the number is an observed test-window effect, an internally modeled portfolio estimate, recognized post-launch revenue, or an external audit.
Explain the methodology. "We separate the randomized test-window estimate from the model that annualizes it. We show uncertainty, eligible traffic, unit economics, adoption, and persistence assumptions."
Show the economics. "The fully loaded program cost is approximately $X. The modeled impact range is $Y–$Z under these assumptions, and recognized post-launch contribution is tracked separately."
Compare like with like. Put randomized experiments, geo tests, lift studies, attribution models, and marketing-mix models on the same page with their scope and limitations. Do not collapse them into one certainty score.
Make the ask. "With additional investment of $X, we can expand into these qualified opportunities. Here is the backlog, the capacity constraint, and a scenario range—not a promised return."
The CFO will ask tough questions. That's their job. But the rigor of the methodology gives you answers that other functions can't provide. Lean into that.
Persistence Is an Assumption to Test
A shipped change may keep working, decay, interact with later releases, or be removed. Seasonality, traffic mix, competitor behavior, and product changes can all alter the effect.
That is why a lifetime-value model needs an explicit persistence curve and why high-impact changes deserve post-launch monitoring or a holdout when feasible. A prior-period portfolio estimate should not simply be carried forward as new revenue. Later periods need their own evidence.
The strongest budget argument is not permanence. It is a transparent chain from test-window evidence to modeled impact to recognized post-launch outcomes.
The PRISM method shows how that chain fits into a test workflow. The NRG program retrospective and SVB geo-test retrospective show how the evidence labels change across different designs.
_Ready to make the experiment decision defensible? GrowthLayer's unified A/B test calculator keeps the plan, statistical readout, and next action together._
FAQ
Can experimentation ROI and brand ROI use the same formula?
They can share a financial framework, but the evidence designs differ. Compare causal confidence, time horizon, coverage, cost, and the decision each estimate supports.
Is there a reliable average percentage of marketing budget wasted?
Not from the evidence reviewed here. Nielsen reported that only 32% of marketers measured traditional and digital media holistically, but that does not convert into a universal waste percentage.
What is the best first step for a budget review?
Create an evidence ledger that separates observed outcomes, modeled contribution, and unmeasured spend. Contact me if you need help defining the ledger and decision rules for one priority funnel.