Articles exploring optimizely through the lens of behavioral science and experimentation. Practical frameworks for growth leaders who measure in revenue, not vanity metrics.
26 articles
The correct Optimizely setup sequence — snippet installation, A/A testing, custom events, naming conventions, and the 5 mistakes that create months of bad data.
The front door to the Optimizely Practitioner Toolkit. Find the right learning path based on where you are, avoid the 5 most common mistakes, and access all 24 practitioner guides organized by cluster.
Visitor-based vs session-based conversion counting, the exact math showing how it changes your reported rate, unique vs all conversions, how to audit your setup, and common bugs that inflate conversions.
The three Optimizely metric types explained for practitioners — when revenue per visitor beats revenue per purchase, the variance problem with revenue metrics, ratio metric gotchas, and practical setup examples.
Why you can only have one primary metric, how to choose it correctly, why revenue per visitor usually beats CVR alone, and how metric selection affects test duration and statistical validity.
The exact technical difference between URL targeting and audience targeting in Optimizely, when to use each, wildcard patterns, regex examples, and the most common targeting mistakes that corrupt test data.
A practitioner-level guide to Optimizely audience conditions — AND/OR logic, cookie targeting, dynamic evaluation timing traps, and why your audience is always smaller than you think.
Most testing roadmaps are just feature wishlists. Here's how to build a real experimentation roadmap—with prioritization frameworks, sequencing logic, and stakeholder buy-in tactics that keep the program moving.
"Conversion rate" means completely different things for an ecommerce site vs. SaaS vs. media company. Here's the right metric hierarchy for each revenue model, with Optimizely setup instructions and worked examples.
Your CEO doesn't care about statistical significance. Here's the one-page results template, the revenue translation formula, and how to handle every awkward stakeholder question about your experiment results.
"Let's test a bigger CTA" is not a hypothesis. Here's the exact structure for writing A/B test hypotheses that produce useful results whether they win or lose—with 5 real rewrites and a hypothesis library framework.
Optimizely and GA4 will never show identical numbers — and that's expected. This guide explains the 5 root causes of discrepancies, how to audit each one, and when a discrepancy signals a real problem vs. normal variance.
Stopping an A/B test at the wrong time — too early or too late — is one of the most expensive mistakes in experimentation. Here are the rules that mature testing programs actually use.
The top-line result is often a lie. This guide shows you how to segment Optimizely results correctly, which segments actually matter, and how to avoid the statistical traps that turn exploratory data into false conclusions.
A practitioner's guide to every element on the Optimizely results page — what it means, what to check first, and how to avoid the most common misreads that lead to bad decisions.
Most teams skip A/A tests and only realize the mistake after shipping a 'winner' that quietly reverses. Here's what an A/A test actually validates, how to interpret results, and the 5 things it catches that you'd never find otherwise.
Not all A/B tests are equal. Here are 10 experiments with tight behavioral hypotheses, realistic lift expectations, and the exact failure modes to watch out for — plus 3 bonus tests the standard lists always miss.
The wrong test type is one of the most common ways CRO programs waste months. Here's the decision framework — with real traffic numbers — for choosing between A/B, MVT, and multi-page experiments in Optimizely.
Someone changed your live A/B test. Maybe it was you. Here's exactly what that broke, why the data is compromised, and the step-by-step rescue workflow to salvage the situation.
Seven years running 100+ experiments taught me that test duration is the most violated rule in CRO. Here's the full framework — minimum days, sample size math, and when it's actually OK to stop early.
MDE isn't a calculator input — it's the foundation of your entire experiment design. Set it wrong and you'll either run 6-month tests or miss real effects entirely. Here's the framework, the math, and the business ROI approach that changes how you plan experiments.
Optimizely now offers three statistical engines: Sequential (Stats Engine), Frequentist Fixed Horizon, and Bayesian. Each one changes what you measure, how you decide, and when you can stop. Here's how to pick the right one for your team.
Most practitioners misread statistical significance as 'probability you're right.' It isn't. Here's what it actually means, why 95% is a convention not a law, how peeking kills your results, and how Optimizely's Stats Engine solves the classical problem.
Tuesday your experiment shows 94% confidence. Friday it's 71%. Nothing changed — so what's happening? Here's how to tell normal statistical fluctuation from novelty effect, regression to the mean, and actual drift.
Running 20 tests at 95% confidence means you expect at least one false positive by chance. Here's how Optimizely's FDR control works, what it doesn't protect against, and how to structure your testing program to minimize spurious wins.
Week 4. Your test shows a 6% lift. Optimizely says 'not enough data.' Here are the five reasons experiments stall before reaching statistical significance — and what to do about each one.