Experimentation Teams Should Operate Like Internal Consultants
How an internal-consulting model changes an experimentation team’s role, with outcomes to measure rather than a promised performance advantage.
Know what to test, when to trust the result, and what to do next. Practical guides for analysts, growth teams, and founders.
How an internal-consulting model changes an experimentation team’s role, with outcomes to measure rather than a promised performance advantage.
Lessons from leading marketing analytics at SVB and NRG, including measurement systems, geo tests, and executive decisions.
SVB OOH geo test: treatment markets grew, but the Seattle control grew faster, so the result did not establish causal lift.
A July 2026 Jobsolv snapshot: 35K+ users, $80K+ total revenue, $0 current paid acquisition, earlier paid tests, and explicit cost limits.
How Atticus Li managed a 27-person Jobsolv roster across the build, with up to 22 active in a week, while working full-time at NRG Energy.
How Atticus Li governed 100+ annual tests across five NRG Energy retail brands, with explicit rules for stopping, collisions, prioritization, and evidence.
How to write problem-first hypotheses that connect observed user friction, a plausible mechanism, and a measurable business decision.
A practical guide for PMs working with experimentation teams, from test duration and scope to interpretation and rollout decisions.
How to connect early experimentation evidence to business decisions without treating annualized models as realized revenue.
A founder-reported Jobsolv cohort: 24 of 26 service clients received at least one interview within 30 days, followed by the transition from service to software.
A practical solo-founder playbook for experimentation, channel choices, and shipping when time, traffic, and budget are constrained.
How to present test-window evidence, modeled financial impact, and recognized outcomes so finance can inspect an experimentation investment.
How durable templates, decision rules, and ownership standards preserve experimentation quality through growth and turnover.
UX researchers trained in academic rigor often struggle to deliver inside real companies.
Five process failures that undermine A/B tests, plus practical fixes for tracking, metrics, sample size, and stakeholder decisions.
The biggest CRO influencers run programs at companies with Netflix-level traffic. That is not your reality.
How a cleaner homepage, a modal instead of a dedicated page, and a flat primary metric quietly killed enrollment conversions—and what to change in your…
A five-step framework for connecting experiment decisions to revenue assumptions, implementation quality, and post-test evidence.
A/B testing is evolving fast. Explore how AI, automation, and new statistical methods will reshape experimentation in the coming years.
A/B testing raises real ethical questions about consent, manipulation, and fairness. Learn where the ethical boundaries are and how to test responsibly.
Machine learning and A/B testing are complementary, not competing. Learn how ML improves experiment design, analysis, and the speed of optimization cycles.
CUPED uses pre-experiment data to reduce noise in your A/B tests. Learn how this variance reduction technique works and when it dramatically improves power.
Pricing experiments are high-stakes and high-reward. Learn the frameworks and safeguards that let you test pricing without damaging trust or revenue.
Low traffic does not mean you cannot experiment. Learn proven strategies for running meaningful A/B tests when your sample size is limited.
Know what to test, when to trust the result, and what to do next. Practical decision guides for analysts, growth teams, and founders. Free. Weekly.
Opens Substack to confirm your subscription · Free · Unsubscribe anytime