What Can a Website Heatmap Reveal About the Wrong Homepage?
A heatmap showed most homepage visitors ignored the extra pathways offered to them. Removing those paths, not adding more, won.
Read Article
→
Blog · Tag
Analytics
Articles exploring analytics through the lens of behavioral science and experimentation. Practical frameworks for growth leaders who measure in revenue, not vanity metrics.
34 articles
Non-Inferiority Testing for SaaS Revenue Changes
Some SaaS changes should raise revenue. Others should simply not break it. A billing flow rewrite, navigation cleanup, design system migration, or applied
Read Article
→
A/B Test Ramp Plans for Revenue-Risk Experiments
A winning test can still lose you money. I see this a lot in high-stakes A/B testing. The team has a solid hypothesis, clean analytics, and good intent,
Read Article
→
Survival Analysis SaaS: Managing Tests With Delayed Revenue
You ran the test. Signups moved. Activation moved. Revenue did not. At least not yet. This is where many SaaS teams make an expensive mistake.
Read Article
→
How I Handle Revenue Outliers in A/B Tests
A test can look like a winner because three customers showed up with a corporate card. I've seen teams ship bad changes, celebrate the lift, then spend a
Read Article
→
Ratio Metrics in A/B Testing Without Misreading Revenue Lift
A test can lift conversion and still hurt revenue. I have watched teams ship winners that looked great in the dashboard but weak in the finance review.
Read Article
→
Exposure Logging Attribution Makes A/B Test Results Trustworthy
Most bad A/B test calls are not statistics problems. They are measurement problems. I see the same mistake over and over.
Read Article
→
The Decision Latency Metric for Experimentation Teams
A test doesn't create value when the chart turns green. It creates value when somebody decides. I've seen teams run clean experiments, get solid analytics,
Read Article
→
Activation Metrics: How to Pick the One That Predicts Retention
Most teams track the wrong activation metric. A practitioner's guide to choosing an activation metric that statistically predicts retention, instrumenting…
Read Article
→
Time-on-Page as a Friction Signal: When Faster Is Actually Better
Interpret time-on-page with conversion, scroll depth, and interaction data to separate faster decisions from abandonment or confusion.
Read Article
→
The Experimentation Win Rate Myth: Why Context Beats a Benchmark
How to report experiment win rates with a visible denominator, decision rule, evidence quality, business impact, and explicit limits.
Read Article
→
Post-Hoc Metric Shopping Is Killing Your Experimentation Program
Why choosing metrics after seeing results creates false confidence, and how pre-commitment protects an experimentation program.
Read Article
→
Vanity Metrics vs. Revenue Metrics: What Your A/B Tests Should Actually Measure
How to distinguish diagnostic engagement metrics from decision metrics and document the path from an experiment to revenue evidence.
Read Article
→
What I Tell New Analysts Who Think 50/50 Splits Need to Be Perfect
Why imperfect A/B traffic splits are normal, when to suspect sample ratio mismatch, and which diagnostic and follow-up checks to run.
Read Article
→
Why Every Company Has a Different Data Dictionary (And Why That Matters for Experimentation)
How to audit the real definitions behind users, sessions, conversions, and revenue before trusting an experiment result.
Read Article
→
Why Your "Unique Visitors" Funnel Is Lying To You (And What To Do Instead)
When breakdown rows exceed total users, you're seeing overlapping populations, not a funnel. Here's why dashboards fail and how to fix it.
Read Article
→
How AI Fit an NRG Workflow That Became Roughly 40% Faster
A first-person NRG estimate: analysis moved from roughly eight to five hours after AI-assisted steps, without isolating AI as the cause.
Read Article
→
Data Storytelling: How to Present Analytics to Executives Who've Never Seen the Data
Lessons from presenting experimentation evidence to executives at NRG and SVB, from decision framing to financial assumptions.
Read Article
→
The EBITDA Impact Model: How I Tied Every Experiment to Business Outcomes
How Atticus Li used NRG Energy's internal EBITDA impact model to translate test-window evidence into assumption-labeled financial estimates.
Read Article
→
From SVB to NRG: What Enterprise Marketing Analytics Actually Looks Like
Lessons from leading marketing analytics at SVB and NRG, including measurement systems, geo tests, and executive decisions.
Read Article
→
Why Most A/B Tests Fail (And How to Fix Your Experimentation Program)
Five process failures that undermine A/B tests, plus practical fixes for tracking, metrics, sample size, and stakeholder decisions.
Read Article
→
The Instrumentation Problem: When Your Tracking Is the Bug
Bad tracking corrupts A/B test results silently. Learn how to detect and prevent instrumentation bugs that make your experiment data unreliable or misleading.
Read Article
→
How to Handle Multiple Metrics in A/B Testing
When A/B tests track multiple metrics, statistical complexity increases. Learn frameworks for managing metric conflicts and making sound decisions.
Read Article
→
Choosing the Right Primary Metric for Your A/B Test
Your primary metric determines whether an A/B test succeeds or fails. Learn how to select metrics that are sensitive, aligned, and actionable.
Read Article
→
How to Create AI-Powered Dashboards Without Writing SQL
Create AI-powered dashboards without writing SQL using natural language queries, automated visualizations, and real-time data connections.
Read Article
→
How NRG Scaled to 100+ Experiments with $30M+ in Internal Program Reporting
How NRG scaled from roughly 20 to 100+ annual tests across five brands, with $30M+ in internal program reporting and explicit evidence limits.
Read Article
→
How Optimizely Counts Conversions (And Why It Changes Your Results)
Visitor-based vs session-based conversion counting, the exact math showing how it changes your reported rate, unique vs all conversions, how to audit your…
Read Article
→
Ratio Metrics, Revenue Metrics, and Conversion Metrics in Optimizely: When to Use Each
The three Optimizely metric types explained for practitioners — when revenue per visitor beats revenue per purchase, the variance problem with revenue…
Read Article
→
Choosing the Right Primary Metric for Your A/B Test
Why you can only have one primary metric, how to choose it correctly, why revenue per visitor usually beats CVR alone, and how metric selection affects test…
Read Article
→
Metrics by Revenue Model: What to Measure Based on Your Business Type
"Conversion rate" means completely different things for an ecommerce site vs. SaaS vs. media company.
Read Article
→
Why Optimizely Data Doesn't Match Google Analytics (And How to Fix It)
Optimizely and GA4 will never show identical numbers — and that's expected.
Read Article
→
Segmenting A/B Test Results in Optimizely: Where the Real Insights Hide
The top-line result is often a lie. This guide shows you how to segment Optimizely results correctly, which segments actually matter, and how to avoid the…
Read Article
→
6 Research Methods That Fuel High-Impact A/B Tests
Discover the six research methods that separate high-impact A/B tests from random guessing.
Read Article
→
Confirmation Bias in Dashboard Design: Why Teams Only See What They Want to See
Dashboard design inadvertently reinforces confirmation bias by making favorable metrics prominent and burying contradictory signals.
Read Article
→