Experiment Portfolio Management
The strategic allocation of testing resources across different risk levels, business areas, and learning objectives — treating the experiment program as an investment portfolio.
What Is Experiment Portfolio Management?
Experiment portfolio management applies investment portfolio theory to experimentation. Just as a financial portfolio balances high-risk/high-reward investments with stable returns, an experiment portfolio balances bold exploratory tests with safer optimization tests. The goal is maximizing total learning and impact, not just individual test success rates.
A balanced portfolio typically includes 70% optimization tests, 20% exploratory tests, and 10% moonshot tests.
Also Known As
- Marketing: Marketing test portfolio, campaign balance
- Sales: Sales experiment portfolio
- Growth: Growth portfolio, experimentation portfolio
- Product: Product bet portfolio, feature experiment mix
- Engineering: Engineering experiment portfolio
- Data: Analysis portfolio, learning portfolio
How It Works
A growth team reviewing Q3 plans finds their current backlog skews 95% optimization (button tests, copy tests, layout tweaks). Win rate is high (~70%), but cumulative impact has flattened. They rebalance: reduce optimization to 70%, add three exploratory tests on new value propositions (20%), and include one moonshot testing an entirely new pricing model (10%).
Six months later, one moonshot becomes the biggest win of the year — and wouldn't have happened under pure optimization discipline.
Best Practices
- Categorize every test as optimization, exploration, or moonshot.
- Set explicit ratios — 70/20/10 is a reasonable starting point.
- Define success differently by category — moonshots succeed through learning, not just metric wins.
- Review portfolio adherence quarterly.
- Protect moonshot slots — they're the first cut when capacity is tight.
Common Mistakes
- Running 100% optimization tests — produces incremental gains and zero breakthroughs.
- Running too many moonshots — produces lots of learning but no compounding base to build on.
- Treating all tests as equal — portfolio thinking requires explicit categorization.
Industry Context
SaaS/B2B: Portfolio management matters most where test cadence is low — each of 10 tests per quarter needs strategic placement.
Ecommerce/DTC: Large portfolios naturally accommodate all three categories — the risk is optimization bias.
Lead gen: Small programs can still portfolio-manage with 1 moonshot per quarter — the discipline matters more than the absolute count.
The Behavioral Science Connection
Portfolio thinking combats the availability heuristic in experimentation — the tendency to prioritize test ideas based on how easily they come to mind (usually the most recent competitor move or stakeholder request). It also combats loss aversion: teams avoid high-risk tests because the potential "loss" of a failed test looms larger than the potential gain of a breakthrough insight.
Key Takeaway
Portfolio management is how mature programs balance compounding gains with breakthrough discoveries — neither pure optimization nor pure exploration produces sustainable impact.