Behavioral Science Glossary — Behavioral Economics, A/B Testing & CRO

Atticus Li

Reference · Glossary

Behavioral Science
& Experimentation
Glossary

Expert-level definitions from a practitioner who's run the experiments. Not textbook theory — applied behavioral science with real A/B test context.

295Terms Defined

Browse by Category

Behavioral Economics

54 terms

Statistics & Methodology

45 terms

Experimentation Strategy

59 terms

Analytics & Attribution

50 terms

A–Z Index

E

F

F-Pattern ReadingAn eye-tracking-validated reading pattern where users scan web pages in an F-shaped sequence: horizo...UX & Digital Psychology False Discovery RateThe expected proportion of rejected null hypotheses that are actually false positives, controlled by...Statistics & Methodology Feature FlaggingA software development practice that allows teams to enable or disable features for specific user se...Experimentation Strategy Feature FlagsA deployment technique that decouples code release from feature exposure, allowing teams to toggle f...A/B Testing Feature Management PlatformA category of tools (LaunchDarkly, Split, Statsig, GrowthBook, etc.) that enable feature flags, prog...Experimentation Strategy First-Touch AttributionAn attribution model that gives 100% of the conversion credit to the first marketing touchpoint a cu...Analytics & Attribution Fitts's LawA predictive model of human movement stating that the time to reach a target is a function of the di...UX & Digital Psychology Flicker EffectA visible flash of the original page content before the test variant loads, caused by client-side te...A/B Testing Flywheel vs FunnelTwo models of how growth works: the funnel is a one-way path from awareness to purchase; the flywhee...Analytics & Attribution Form OptimizationThe practice of improving web forms to increase completion rates by reducing friction, cognitive loa...Conversion Rate Optimization Fractional Factorial DesignAn experimental design that tests a strategically chosen subset of all possible factor combinations,...A/B Testing Framing EffectA cognitive bias where people react differently to the same information depending on how it is prese...Behavioral Economics Friction PointsAny element of a user experience that slows down, confuses, or prevents a user from completing a des...Conversion Rate Optimization Full Factorial DesignAn experimental design that tests every possible combination of multiple factors and their levels, r...A/B Testing Funnel AnalysisA method of tracking and visualizing the sequential steps users take toward a conversion goal, ident...Analytics & Attribution

G

GA4 ConversionsEvents in GA4 that have been marked as business-critical, enabling their use in Google Ads optimizat...Analytics & Attribution GA4 Event TrackingGoogle Analytics 4's event-based measurement model where every user interaction is captured as an ev...Analytics & Attribution Geo-ExperimentationAn experimental method that uses geographic regions as test and control units to estimate the effect...Analytics & Attribution Gestalt PrinciplesA set of laws describing how the human brain organizes visual elements into groups and patterns, inc...UX & Digital Psychology Goal Gradient EffectThe tendency for people to increase effort as they get closer to completing a goal — acceleration to...Behavioral Economics Google OptimizeA free A/B testing tool from Google that was deprecated in 2023, leaving a gap that paid and open-so...Experimentation Strategy Gross Revenue Retention (GRR)The percentage of recurring revenue retained from an existing customer cohort excluding expansion — ...Analytics & Attribution Growth LoopsSelf-reinforcing cycles where the output of one cycle becomes the input to the next — a more accurat...Analytics & Attribution GrowthBookAn open-source feature flag and experimentation platform that can be self-hosted, with a SaaS option...Experimentation Strategy Guardrail MetricsSecondary metrics monitored during A/B tests to ensure that optimizing the primary metric does not c...A/B Testing Guardrail MetricsProtective metrics monitored during experiments to ensure that improving the target metric doesn't c...Experimentation Strategy

H

Halo EffectA cognitive bias where a positive impression in one area influences perception across unrelated attr...Behavioral Economics Heap AutocaptureHeap's approach of automatically capturing every user interaction — clicks, form fills, pageviews — ...Analytics & Attribution Heap ExperimentsHeap's experimentation capability, leveraging its autocapture analytics to run and analyze A/B tests...Experimentation Strategy Heat MapsVisual representations of user interaction data on a web page, using color gradients to show where u...Conversion Rate Optimization Heterogeneous Treatment Effects (HTE)The observation that treatment effects vary across subgroups — the average effect hides winners and ...Statistics & Methodology Hick's LawThe time it takes to make a decision increases logarithmically with the number and complexity of cho...Behavioral Economics Hindsight BiasThe tendency to believe, after an event has occurred, that one would have predicted or expected the ...Behavioral Economics Holdback TestingAn experimentation strategy where a pre-specified group continues to see the control after a variant...A/B Testing Holdout TestingA method for measuring the cumulative impact of all shipped experiments by withholding changes from ...Experimentation Strategy Horn EffectThe cognitive bias where a single negative trait or impression causes a person to view everything el...Behavioral Economics Hyperbolic DiscountingThe tendency to prefer smaller, immediate rewards over larger, delayed rewards — with the preference...Behavioral Economics Hypothesis TestingThe practice of formulating a specific, falsifiable prediction before running an experiment — the fo...Experimentation Strategy Hypothesis-Driven DevelopmentA product development approach where every feature or change starts with a falsifiable hypothesis ab...Experimentation Strategy

I

ICE Scoring FrameworkA prioritization framework that scores experiment ideas by Impact, Confidence, and Ease — each on a ...Experimentation Strategy IKEA EffectThe cognitive bias where people place disproportionately high value on things they helped create, ev...Behavioral Economics IKEA Effect (Bias)A cognitive bias where people place disproportionately higher value on products or experiences they ...Behavioral Economics Incrementality TestingAn experimental method for estimating the incremental effect of a marketing activity by comparing ou...Analytics & Attribution InstapageAn enterprise landing page platform with A/B testing, personalization, and ad-to-landing-page mappin...Experimentation Strategy Instrumental VariablesA causal inference technique that uses a variable affecting treatment but not outcome directly to id...Statistics & Methodology Interaction EffectsThe phenomenon where multiple simultaneous A/B tests influence each other's results, making it impos...A/B Testing

J

Jakob's LawThe principle that users spend most of their time on other websites and therefore prefer your site t...UX & Digital Psychology Jobs to Be Done (JTBD)A framework arguing that customers don't buy products — they hire products to do a specific job in t...Analytics & Attribution

K

KameleoonA hybrid experimentation and personalization platform with strong server-side, client-side, and feat...Experimentation Strategy

L

Last-Touch AttributionAn attribution model that gives 100% of the conversion credit to the final marketing touchpoint befo...Analytics & Attribution Latin Square DesignAn experimental design that controls for two sources of variability simultaneously by arranging trea...A/B Testing LaunchDarklyAn enterprise feature flag and feature management platform that also supports experimentation on top...Experimentation Strategy Lead ScoringA methodology for ranking prospects by assigning numerical values to behaviors and attributes that i...Conversion Rate Optimization Learning RepositoryA centralized, searchable knowledge base of insights derived from experiments — organized by behavio...Experimentation Strategy Lift AnalysisA measurement technique that quantifies the incremental improvement (lift) in a target metric caused...Analytics & Attribution Loss AversionThe tendency for people to weigh potential losses differently from equivalent gains, with the magnit...Behavioral Economics LTV:CAC RatioThe ratio of customer lifetime value to customer acquisition cost — the fundamental unit-economics t...Analytics & Attribution

M

Machine Learning PersonalizationUsing ML models to select treatments, content, or experiences per user based on predicted incrementa...Statistics & Methodology Macro-ConversionThe primary goal action that directly drives business value, such as a completed purchase, a signed ...Conversion Rate Optimization Magic Number (SaaS)A SaaS efficiency metric estimating how much annualized net-new recurring revenue is generated per d...Analytics & Attribution Mann-Whitney U TestA non-parametric statistical test that compares two independent groups without assuming normal distr...Statistics & Methodology Marketing Mix ModelingA statistical analysis technique that uses historical data to quantify the impact of various marketi...Analytics & Attribution Markov Chain Monte Carlo (MCMC)A family of algorithms that samples from complex probability distributions by constructing a Markov ...Statistics & Methodology Mental AccountingThe cognitive process by which people categorize, evaluate, and track their spending in separate men...Behavioral Economics Mere Exposure EffectThe psychological phenomenon where people develop a preference for things simply because they've bee...Behavioral Economics Meta-Analysis of ExperimentsCombining results from multiple experiments to estimate program-level effects, heterogeneity, and pu...Statistics & Methodology Micro-ConversionA small, measurable action that indicates a user is progressing toward a primary conversion goal, su...Conversion Rate Optimization Miller's Law (7 plus or minus 2)The observation by psychologist George Miller that the average person can hold approximately 7 (plus...UX & Digital Psychology Minimum Detectable Effect (MDE)The smallest true effect an experiment can reliably detect given its sample size, significance level...Statistics & Methodology Minimum Viable ExperimentThe smallest, fastest experiment that can validate or invalidate a hypothesis, stripping away unnece...Experimentation Strategy Mixpanel ExperimentsMixpanel's experimentation capability, enabling A/B testing with metrics defined directly from Mixpa...Experimentation Strategy Mixpanel FunnelsMixpanel's funnel analysis feature for measuring conversion rates between sequential events, with se...Analytics & Attribution Mobile-First DesignA design strategy that begins with the smallest screen size and most constrained environment, then p...UX & Digital Psychology MonetateA personalization and experimentation platform focused on retail and ecommerce, offering testing, AI...Experimentation Strategy Monte Carlo SimulationUsing repeated random sampling to approximate the distribution of outcomes from a probabilistic syst...Statistics & Methodology Multi-Touch AttributionAn attribution framework that distributes conversion credit across multiple marketing touchpoints in...Analytics & Attribution Multiple Comparisons ProblemThe increased probability of falsely identifying a significant result when conducting multiple simul...Statistics & Methodology Multivariate TestingAn experiment that tests multiple variables simultaneously to find the optimal combination, rather t...A/B Testing

N

Net Revenue Retention (NRR)The percentage of recurring revenue retained from an existing customer cohort over a time period, in...Analytics & Attribution Network EffectsA phenomenon where a product becomes more valuable to each user as more users join — the foundationa...Analytics & Attribution Network Effects in ExperimentsInterference between experiment groups that occurs when one user's treatment affects another user's ...A/B Testing Normal DistributionA symmetric, bell-shaped probability distribution defined by its mean and standard deviation, where ...Statistics & Methodology North Star MetricThe single metric that best captures the core value your product delivers to customers, and that the...Analytics & Attribution North Star Metric (Experimentation)The single metric that best captures the core value your product delivers to customers — used to ali...Experimentation Strategy Novelty Effect (in Testing)A temporary lift in engagement or conversion caused by the newness of a change rather than its inher...A/B Testing Nudge (Libertarian Paternalism)An approach to choice architecture that steers people toward better decisions while preserving their...Behavioral Economics Nudge TheoryA concept in behavioral science that proposes designing choice environments to influence decisions w...Behavioral Economics

O

Omission BiasThe tendency to judge harmful actions as worse than equally harmful inactions, leading people to pre...Behavioral Economics OmniconvertA conversion optimization platform combining A/B testing, on-site surveys, and customer segmentation...Experimentation Strategy Optimism BiasThe tendency to overestimate the likelihood of positive events and underestimate the likelihood of n...Behavioral Economics OptimizelyAn enterprise experimentation platform offering A/B testing, feature management, and personalization...Experimentation Strategy

P

Page Speed & ConversionThe relationship between page performance and user behavior, measured in the context of a specific p...Conversion Rate Optimization Pain of PayingThe negative emotional response triggered by spending money, which varies in intensity depending on ...Behavioral Economics Paradox of ChoiceThe counterintuitive finding that having too many options leads to decision paralysis, lower satisfa...Behavioral Economics Paradox of Choice (Psychology)The psychological phenomenon where an abundance of options, rather than liberating, causes anxiety, ...Behavioral Economics Paradox of the Active UserThe observation that users prefer to start using a system immediately rather than reading instructio...UX & Digital Psychology Payback PeriodThe number of months required for a customer's gross profit contribution to recover the cost of acqu...Analytics & Attribution Peak-End RulePeople judge experiences based on the most intense moment (peak) and the final moment (end), not the...Behavioral Economics PersonalizationThe practice of dynamically tailoring web content, offers, and experiences to individual users based...Conversion Rate Optimization PersonyzeA mid-market personalization and A/B testing platform offering behavioral targeting, recommendations...Experimentation Strategy Post-Hoc AnalysisAnalysis performed after an experiment concludes that was not part of the original test plan — usefu...A/B Testing Posterior DistributionIn Bayesian statistics, the updated probability distribution of a parameter after combining prior be...Statistics & Methodology Power AnalysisThe pre-experiment calculation that determines the sample size required to detect a given effect wit...Statistics & Methodology Pre-Registration (Experiment Protocol)The practice of formally documenting an experiment's hypothesis, metrics, sample size, and analysis ...A/B Testing Present BiasThe tendency to overvalue immediate rewards and undervalue future rewards, leading to decisions that...Behavioral Economics Primacy Effect in TestingA bias where existing users resist changes to familiar interfaces, causing a negative initial reacti...A/B Testing Prior DistributionIn Bayesian statistics, the probability distribution representing existing beliefs or knowledge abou...Statistics & Methodology Product AnalyticsThe discipline of analyzing user behavior inside a digital product — distinct from marketing analyti...Analytics & Attribution Product Qualified Lead (PQL)A user who has experienced meaningful value in a free or trial product and demonstrated behaviors su...Analytics & Attribution Product-Led Growth (PLG)A go-to-market strategy where the product itself drives acquisition, activation, retention, and expa...Analytics & Attribution Progressive DisclosureA design strategy that shows only the most essential information and options initially, revealing ad...UX & Digital Psychology Propensity Score MatchingA quasi-experimental technique that matches treated and untreated subjects based on their predicted ...Analytics & Attribution Prospect TheoryA behavioral economics theory describing how people evaluate outcomes relative to a reference point,...Behavioral Economics

R

Rage ClicksA pattern of rapid, repeated clicks on the same element or area, indicating user frustration caused ...Conversion Rate Optimization Randomization UnitThe entity (user, session, pageview, device, or account) that is randomly assigned to a test variant...A/B Testing Rapid Prototyping for ExperimentsThe practice of quickly building low-fidelity test variants to validate concepts before investing in...Experimentation Strategy ReactanceThe psychological resistance that arises when people feel their freedom of choice is being threatene...Behavioral Economics Reciprocity PrincipleThe social norm that compels people to return favors — when you give something of value first, peopl...Behavioral Economics Recognition Over RecallThe cognitive principle that people are better at recognizing previously encountered information tha...UX & Digital Psychology Redirect TestingAn A/B testing method where visitors are redirected to entirely different URLs rather than seeing va...A/B Testing Regression DiscontinuityA quasi-experimental design that identifies causal effects by comparing units just above and just be...Statistics & Methodology Regret AversionThe tendency to avoid making decisions that might lead to regret, often resulting in inaction, risk ...Behavioral Economics Reinforcement Learning for ExperimentationUsing RL algorithms — multi-armed bandits, contextual bandits, and full RL — to adaptively allocate ...Statistics & Methodology Representativeness HeuristicA cognitive shortcut where people judge the probability of something by how closely it matches a pro...Behavioral Economics Retention CurveA graph showing the percentage of a user cohort still active at each time period after signup.Analytics & Attribution Return on Ad Spend (ROAS)The revenue generated for every dollar spent on advertising, calculated as (revenue from ads / ad sp...Conversion Rate Optimization Revenue AttributionThe practice of connecting specific revenue outcomes to the marketing activities, channels, and touc...Analytics & Attribution Revenue Churn vs Logo ChurnTwo distinct churn metrics: logo churn measures the percentage of customers lost; revenue churn meas...Analytics & Attribution Risk Appetite in ExperimentationAn organization's tolerance for potential negative outcomes from experiments, which determines what ...Experimentation Strategy Roll-Out StrategyThe planned approach for gradually deploying a winning A/B test variant to the full user population,...A/B Testing

S

Sales-Led Growth (SLG)A go-to-market strategy where a human sales team is the primary driver of revenue, typically require...Analytics & Attribution Sample Ratio Mismatch (SRM)A diagnostic check that detects when the actual traffic split in an A/B test differs significantly f...A/B Testing SatisficingA decision-making strategy where people choose the first option that meets their minimum acceptable ...Behavioral Economics Scarcity BiasThe tendency to place higher value on things that are scarce or perceived as limited in availability...Behavioral Economics Scroll DepthA measurement of how far down a page users scroll, typically expressed as a percentage of total page...Conversion Rate Optimization Segment (CDP for Experimentation)A Customer Data Platform (now part of Twilio) that collects, unifies, and routes user event data to ...Experimentation Strategy Segmentation in A/B TestsThe practice of analyzing A/B test results across different user subgroups to understand whether tre...A/B Testing Selection BiasA systematic error that occurs when the sample studied is not representative of the population inten...Statistics & Methodology Sequential TestingA statistical methodology for monitoring A/B test results as data accumulates in real time, using ad...A/B Testing Serial Position EffectThe tendency to remember items at the beginning (primacy effect) and end (recency effect) of a list ...Behavioral Economics Server-Side TestingAn experimentation approach where variations are rendered on the server before the page is delivered...A/B Testing Session RecordingsPlayback videos of individual user sessions that capture mouse movements, clicks, scrolls, and page ...Conversion Rate Optimization SignifierA perceivable indicator that communicates what action is possible or how an element should be used, ...UX & Digital Psychology Social ProofThe psychological tendency to look to others' actions and opinions when deciding what to do — especi...Behavioral Economics Split URL TestingAn experimentation method that tests entirely different page URLs or designs against each other by s...A/B Testing Split.ioA feature delivery and experimentation platform focused on engineering teams, with strong support fo...Experimentation Strategy Standard DeviationA measure of the average amount of variability in a dataset, representing how spread out values are ...Statistics & Methodology Statistical PowerThe probability that a test will correctly detect a true effect when one exists — conventionally set...Statistics & Methodology StatsigA modern experimentation and feature management platform with strong data-warehouse integration, a g...Experimentation Strategy Status Quo BiasThe preference for the current state of affairs, where any change from the baseline is perceived as ...Behavioral Economics Stratified SamplingA sampling method that divides a population into distinct subgroups (strata) and samples from each p...Statistics & Methodology Sunk Cost FallacyThe tendency to continue investing in something because of previously invested resources (time, mone...Behavioral Economics Survivorship BiasThe logical error of focusing on entities that passed a selection process while overlooking those th...Statistics & Methodology Synthetic Control MethodA causal inference technique that constructs an artificial control group by weighting untreated unit...Analytics & Attribution System 1 vs System 2 ThinkingThe dual-process theory of cognition where System 1 operates fast, automatic, and intuitive, while S...Behavioral Economics

T

T-TestA statistical hypothesis test that compares the means of two groups to determine if they are signifi...Statistics & Methodology Tealium iQAn enterprise tag management system (TMS) commonly used to deploy experimentation, analytics, and ma...Experimentation Strategy Test & Learn CultureAn organizational mindset where decisions are validated through experiments rather than authority, a...Experimentation Strategy Test DemocratizationThe practice of enabling non-specialist teams across the organization to design, launch, and analyze...Experimentation Strategy Test DurationThe length of time an A/B test should run to produce valid results — balancing the need for sufficie...A/B Testing Test Stopping RulesPre-defined criteria that specify when an A/B test should be concluded, preventing both premature st...A/B Testing Test VelocityThe rate at which an organization launches, concludes, and learns from experiments — a key indicator...A/B Testing Thompson SamplingA Bayesian bandit algorithm that allocates traffic to variants based on the probability that each va...A/B Testing Thumb Zone DesignThe practice of placing interactive elements within the area of a mobile screen that is comfortably ...UX & Digital Psychology Time to Value (TTV)The elapsed time from a user's first interaction with a product until they experience its core value...Analytics & Attribution Type I Error (False Positive)Concluding that a variant beat control when in reality there is no true effect — the risk is control...Statistics & Methodology Type II Error (False Negative)Failing to detect a real effect — the risk is controlled by statistical power and equals 1 minus pow...Statistics & Methodology

U

UnbounceA landing page builder with integrated A/B testing, focused on marketers running paid traffic campai...Experimentation Strategy Uplift ModelingA machine learning approach that predicts the causal effect of a treatment on an individual user — w...Statistics & Methodology User Journey MappingThe process of visualizing the complete end-to-end experience a user has with a product or service, ...Conversion Rate Optimization UTM ParametersStandardized URL query parameters (source, medium, campaign, term, content) appended to links to tra...Analytics & Attribution

V

Value Proposition TestingThe systematic process of testing different articulations of your product or service value against e...Conversion Rate Optimization VarianceThe average of the squared differences between each data point and the mean, measuring the spread of...Statistics & Methodology Variance Reduction TechniquesMethods that shrink the noise in experiment metrics — including CUPED, stratification, post-stratifi...Statistics & Methodology Variant DesignThe process of creating the experimental treatment in an A/B test, translating a behavioral hypothes...A/B Testing Viral Coefficient (K-Factor)The average number of new users a single existing user generates through referrals or sharing during...Analytics & Attribution Visual HierarchyThe arrangement of design elements in order of importance using size, color, contrast, spacing, and ...UX & Digital Psychology Von Restorff Effect (Isolation Effect)The tendency for an item that stands out as distinctly different from its surroundings to be more ea...Behavioral Economics VWO (Visual Website Optimizer)A CRO-focused experimentation platform that combines A/B testing with behavioral analytics tools lik...Experimentation Strategy

W

Webtrends OptimizeA UK-based experimentation and personalization platform offering A/B testing, server-side testing, a...Experimentation Strategy Winner DeterminationThe process of deciding which variant in an A/B test has genuinely outperformed the control, using s...A/B Testing

Z

Z-Pattern LayoutA visual scanning pattern where the eye moves in a Z-shape across a page: left to right across the t...UX & Digital Psychology Z-ScoreA standardized score indicating how many standard deviations a data point or test statistic is from ...Statistics & Methodology Zeigarnik EffectThe psychological tendency to remember incomplete or interrupted tasks better than completed ones, c...Behavioral Economics Zero Risk BiasThe preference to completely eliminate one risk rather than achieving a larger overall reduction in ...Behavioral Economics