Experiment Portfolio Management
The strategic allocation of testing resources across different risk levels, business areas, and learning objectives — treating the experiment program as an investment portfolio.
What Is Experiment Portfolio Management?
Experiment portfolio management applies investment portfolio theory to experimentation. Just as a financial portfolio balances high-risk/high-reward investments with stable returns, an experiment portfolio balances bold exploratory tests with safer optimization tests. The goal is maximizing total learning and impact, not just individual test success rates.
A 70/20/10 split can be an illustrative starting point, not a benchmark. Set the mix from risk tolerance, learning goals, traffic, and strategic uncertainty.
Also Known As
- Marketing: Marketing test portfolio, campaign balance
- Sales: Sales experiment portfolio
- Growth: Growth portfolio, experimentation portfolio
- Product: Product bet portfolio, feature experiment mix
- Engineering: Engineering experiment portfolio
- Data: Analysis portfolio, learning portfolio
How It Works
Illustrative planning exercise: a team reviews whether its backlog overweights incremental optimization and chooses a mix of optimization, exploratory, and higher-uncertainty tests. Evaluate the resulting portfolio by learning value, decision quality, risk, and observed impact rather than a target win rate.
Best Practices
- Categorize every test as optimization, exploration, or moonshot.
- Set explicit ratios — 70/20/10 is a reasonable starting point.
- Define success differently by category — moonshots succeed through learning, not just metric wins.
- Review portfolio adherence quarterly.
- Protect moonshot slots — they're the first cut when capacity is tight.
Common Mistakes
- Running 100% optimization tests — produces incremental gains and zero breakthroughs.
- Running too many moonshots — produces lots of learning but no compounding base to build on.
- Treating all tests as equal — portfolio thinking requires explicit categorization.
Industry Context
SaaS/B2B: Portfolio management matters most where test cadence is low — each of 10 tests per quarter needs strategic placement.
Ecommerce/DTC: Large portfolios naturally accommodate all three categories — the risk is optimization bias.
Lead gen: Small programs can still portfolio-manage with 1 moonshot per quarter — the discipline matters more than the absolute count.
The Behavioral Science Connection
Portfolio thinking combats the availability heuristic in experimentation — the tendency to prioritize test ideas based on how easily they come to mind (usually the most recent competitor move or stakeholder request). It also combats loss aversion: teams avoid high-risk tests because the potential "loss" of a failed test looms larger than the potential gain of a breakthrough insight.
Key Takeaway
Portfolio management is how mature programs balance compounding gains with breakthrough discoveries — neither pure optimization nor pure exploration produces sustainable impact.