Articles exploring analytics through the lens of behavioral science and experimentation. Practical frameworks for growth leaders who measure in revenue, not vanity metrics.
33 articles
Some SaaS changes should raise revenue. Others should simply not break it. A billing flow rewrite, navigation cleanup, design system migration, or applied
A winning test can still lose you money. I see this a lot in high-stakes A/B testing. The team has a solid hypothesis, clean analytics, and good intent,
You ran the test. Signups moved. Activation moved. Revenue did not. At least not yet. This is where many SaaS teams make an expensive mistake.
A test can look like a winner because three customers showed up with a corporate card. I've seen teams ship bad changes, celebrate the lift, then spend a
A test can lift conversion and still hurt revenue. I have watched teams ship winners that looked great in the dashboard but weak in the finance review.
Most bad A/B test calls are not statistics problems. They are measurement problems. I see the same mistake over and over.
A test doesn't create value when the chart turns green. It creates value when somebody decides. I've seen teams run clean experiments, get solid analytics,
Most teams track the wrong activation metric. A practitioner's guide to choosing an activation metric that statistically predicts retention, instrumenting…
Time-on-page is one of the most-misread metrics in CRO. Faster sometimes means more friction (users gave up) and sometimes means less friction (users…
Most A/B tests lose. Industry win rates hover around 15-30%, and that's not a failure — it's how experimentation actually works.
The pressure to prove every test is a winner leads teams to cherry-pick metrics after seeing results.
CTR, scroll depth, and time on page don't pay the bills. Here's how to tie every experiment to actual revenue — and why most programs measure the wrong things.
Every new analyst panics when their A/B test shows 51/49 instead of 50/50.
GA4 and Adobe Analytics don't even agree on what a 'user' is. Every company's data dictionary has quirks that can silently corrupt your experiment results…
When breakdown rows exceed total users, you're seeing overlapping populations, not a funnel. Here's why dashboards fail and how to fix it.
Atticus Li reduced experimentation analysis time by 40% at NRG Energy by integrating AI tools including Claude, ChatGPT, and Optimizely AI into the testing…
Atticus Li shares data storytelling lessons from presenting experimentation results to C-suite executives at NRG Energy and Silicon Valley Bank — including…
Atticus Li designed NRG Energy's EBITDA impact estimation model that translates A/B test results into verified financial impact, turning experimentation…
Atticus Li shares lessons from leading marketing analytics at Silicon Valley Bank and NRG Energy — covering Google Analytics vs Adobe Analytics, data…
Most A/B tests fail because the process is broken, not because the ideas are bad.
Bad tracking corrupts A/B test results silently. Learn how to detect and prevent instrumentation bugs that make your experiment data unreliable or misleading.
When A/B tests track multiple metrics, statistical complexity increases. Learn frameworks for managing metric conflicts and making sound decisions.
Your primary metric determines whether an A/B test succeeds or fails. Learn how to select metrics that are sensitive, aligned, and actionable.
Create AI-powered dashboards without writing SQL using natural language queries, automated visualizations, and real-time data connections.
Atticus Li shares how he scaled NRG Energy's experimentation program from 20 tests per year to 150+ total experiments across 7 brands, tying every test to…
Visitor-based vs session-based conversion counting, the exact math showing how it changes your reported rate, unique vs all conversions, how to audit your…
The three Optimizely metric types explained for practitioners — when revenue per visitor beats revenue per purchase, the variance problem with revenue…
Why you can only have one primary metric, how to choose it correctly, why revenue per visitor usually beats CVR alone, and how metric selection affects test…
"Conversion rate" means completely different things for an ecommerce site vs. SaaS vs. media company.
Optimizely and GA4 will never show identical numbers — and that's expected.
The top-line result is often a lie. This guide shows you how to segment Optimizely results correctly, which segments actually matter, and how to avoid the…
Discover the six research methods that separate high-impact A/B tests from random guessing.
Dashboard design inadvertently reinforces confirmation bias by making favorable metrics prominent and burying contradictory signals.