Behavioral Science Glossary — Behavioral Economics, A/B Testing & CRO

Atticus Li

Reference · Glossary

Behavioral Science
& Experimentation
Glossary

Expert-level definitions from a practitioner who's run the experiments. Not textbook theory — applied behavioral science with real A/B test context.

295 Terms Defined

Browse by Category

Behavioral Economics

54 terms

A/B Testing

37 terms

Statistics & Methodology

45 terms

Experimentation Strategy

59 terms

Analytics & Attribution

50 terms

A–Z Index

E

F

F-Pattern Reading An eye-tracking-validated reading pattern where users scan web pages in an F-shaped sequence: horizo ... UX & Digital Psychology False Discovery Rate The expected proportion of rejected null hypotheses that are actually false positives, controlled by ... Statistics & Methodology Feature Flagging A software development practice that allows teams to enable or disable features for specific user se ... Experimentation Strategy Feature Flags A deployment technique that decouples code release from feature exposure, allowing teams to toggle f ... A/B Testing Feature Management Platform A category of tools (LaunchDarkly, Split, Statsig, GrowthBook, etc.) that enable feature flags, prog ... Experimentation Strategy First-Touch Attribution An attribution model that gives 100% of the conversion credit to the first marketing touchpoint a cu ... Analytics & Attribution Fitts's Law A predictive model of human movement stating that the time to reach a target is a function of the di ... UX & Digital Psychology Flicker Effect A visible flash of the original page content before the test variant loads, caused by client-side te ... A/B Testing Flywheel vs Funnel Two models of how growth works: the funnel is a one-way path from awareness to purchase; the flywhee ... Analytics & Attribution Form Optimization The practice of improving web forms to increase completion rates by reducing friction, cognitive loa ... Conversion Rate Optimization Fractional Factorial Design An experimental design that tests a strategically chosen subset of all possible factor combinations, ... A/B Testing Framing Effect A cognitive bias where people react differently to the same information depending on how it is prese ... Behavioral Economics Friction Points Any element of a user experience that slows down, confuses, or prevents a user from completing a des ... Conversion Rate Optimization Full Factorial Design An experimental design that tests every possible combination of multiple factors and their levels, r ... A/B Testing Funnel Analysis A method of tracking and visualizing the sequential steps users take toward a conversion goal, ident ... Analytics & Attribution

G

GA4 Conversions Events in GA4 that have been marked as business-critical, enabling their use in Google Ads optimizat ... Analytics & Attribution GA4 Event Tracking Google Analytics 4's event-based measurement model where every user interaction is captured as an ev ... Analytics & Attribution Geo-Experimentation An experimental method that uses geographic regions as test and control units to measure the causal ... Analytics & Attribution Gestalt Principles A set of laws describing how the human brain organizes visual elements into groups and patterns, inc ... UX & Digital Psychology Goal Gradient Effect The tendency for people to increase effort as they get closer to completing a goal — acceleration to ... Behavioral Economics Google Optimize A free A/B testing tool from Google that was deprecated in 2023, leaving a gap that paid and open-so ... Experimentation Strategy Gross Revenue Retention (GRR) The percentage of recurring revenue retained from an existing customer cohort excluding expansion — ... Analytics & Attribution Growth Loops Self-reinforcing cycles where the output of one cycle becomes the input to the next — a more accurat ... Analytics & Attribution GrowthBook An open-source feature flag and experimentation platform that can be self-hosted, with a SaaS option ... Experimentation Strategy Guardrail Metrics Secondary metrics monitored during A/B tests to ensure that optimizing the primary metric does not c ... A/B Testing Guardrail Metrics Protective metrics monitored during experiments to ensure that improving the target metric doesn't c ... Experimentation Strategy

H

Halo Effect A cognitive bias where a positive impression in one area influences perception across unrelated attr ... Behavioral Economics Heap Autocapture Heap's approach of automatically capturing every user interaction — clicks, form fills, pageviews — ... Analytics & Attribution Heap Experiments Heap's experimentation capability, leveraging its autocapture analytics to run and analyze A/B tests ... Experimentation Strategy Heat Maps Visual representations of user interaction data on a web page, using color gradients to show where u ... Conversion Rate Optimization Heterogeneous Treatment Effects (HTE) The observation that treatment effects vary across subgroups — the average effect hides winners and ... Statistics & Methodology Hick's Law The time it takes to make a decision increases logarithmically with the number and complexity of cho ... Behavioral Economics Hindsight Bias The tendency to believe, after an event has occurred, that one would have predicted or expected the ... Behavioral Economics Holdback Testing An experimentation strategy where a small percentage of users permanently see the control experience ... A/B Testing Holdout Testing A method for measuring the cumulative impact of all shipped experiments by withholding changes from ... Experimentation Strategy Horn Effect The cognitive bias where a single negative trait or impression causes a person to view everything el ... Behavioral Economics Hyperbolic Discounting The tendency to prefer smaller, immediate rewards over larger, delayed rewards — with the preference ... Behavioral Economics Hypothesis Testing The practice of formulating a specific, falsifiable prediction before running an experiment — the fo ... Experimentation Strategy Hypothesis-Driven Development A product development approach where every feature or change starts with a falsifiable hypothesis ab ... Experimentation Strategy

I

ICE Scoring Framework A prioritization framework that scores experiment ideas by Impact, Confidence, and Ease — each on a ... Experimentation Strategy IKEA Effect The cognitive bias where people place disproportionately high value on things they helped create, ev ... Behavioral Economics IKEA Effect (Bias) A cognitive bias where people place disproportionately higher value on products or experiences they ... Behavioral Economics Incrementality Testing An experimental method that measures the true causal impact of a marketing activity by comparing out ... Analytics & Attribution Instapage An enterprise landing page platform with A/B testing, personalization, and ad-to-landing-page mappin ... Experimentation Strategy Instrumental Variables A causal inference technique that uses a variable affecting treatment but not outcome directly to id ... Statistics & Methodology Interaction Effects The phenomenon where multiple simultaneous A/B tests influence each other's results, making it impos ... A/B Testing

J

Jakob's Law The principle that users spend most of their time on other websites and therefore prefer your site t ... UX & Digital Psychology Jobs to Be Done (JTBD) A framework arguing that customers don't buy products — they hire products to do a specific job in t ... Analytics & Attribution

K

Kameleoon A hybrid experimentation and personalization platform with strong server-side, client-side, and feat ... Experimentation Strategy

L

Last-Touch Attribution An attribution model that gives 100% of the conversion credit to the final marketing touchpoint befo ... Analytics & Attribution Latin Square Design An experimental design that controls for two sources of variability simultaneously by arranging trea ... A/B Testing LaunchDarkly An enterprise feature flag and feature management platform that also supports experimentation on top ... Experimentation Strategy Lead Scoring A methodology for ranking prospects by assigning numerical values to behaviors and attributes that i ... Conversion Rate Optimization Learning Repository A centralized, searchable knowledge base of insights derived from experiments — organized by behavio ... Experimentation Strategy Lift Analysis A measurement technique that quantifies the incremental improvement (lift) in a target metric caused ... Analytics & Attribution Loss Aversion The psychological tendency for people to prefer avoiding losses over acquiring equivalent gains — lo ... Behavioral Economics LTV:CAC Ratio The ratio of customer lifetime value to customer acquisition cost — the fundamental unit-economics t ... Analytics & Attribution

M

Machine Learning Personalization Using ML models to select treatments, content, or experiences per user based on predicted incrementa ... Statistics & Methodology Macro-Conversion The primary goal action that directly drives business value, such as a completed purchase, a signed ... Conversion Rate Optimization Magic Number (SaaS) A SaaS efficiency metric calculating how much ARR is generated per dollar of sales and marketing spe ... Analytics & Attribution Mann-Whitney U Test A non-parametric statistical test that compares two independent groups without assuming normal distr ... Statistics & Methodology Marketing Mix Modeling A statistical analysis technique that uses historical data to quantify the impact of various marketi ... Analytics & Attribution Markov Chain Monte Carlo (MCMC) A family of algorithms that samples from complex probability distributions by constructing a Markov ... Statistics & Methodology Mental Accounting The cognitive process by which people categorize, evaluate, and track their spending in separate men ... Behavioral Economics Mere Exposure Effect The psychological phenomenon where people develop a preference for things simply because they've bee ... Behavioral Economics Meta-Analysis of Experiments Combining results from multiple experiments to estimate program-level effects, heterogeneity, and pu ... Statistics & Methodology Micro-Conversion A small, measurable action that indicates a user is progressing toward a primary conversion goal, su ... Conversion Rate Optimization Miller's Law (7 plus or minus 2) The observation by psychologist George Miller that the average person can hold approximately 7 (plus ... UX & Digital Psychology Minimum Detectable Effect (MDE) The smallest true effect an experiment can reliably detect given its sample size, significance level ... Statistics & Methodology Minimum Viable Experiment The smallest, fastest experiment that can validate or invalidate a hypothesis, stripping away unnece ... Experimentation Strategy Mixpanel Experiments Mixpanel's experimentation capability, enabling A/B testing with metrics defined directly from Mixpa ... Experimentation Strategy Mixpanel Funnels Mixpanel's funnel analysis feature for measuring conversion rates between sequential events, with se ... Analytics & Attribution Mobile-First Design A design strategy that begins with the smallest screen size and most constrained environment, then p ... UX & Digital Psychology Monetate A personalization and experimentation platform focused on retail and ecommerce, offering testing, AI ... Experimentation Strategy Monte Carlo Simulation Using repeated random sampling to approximate the distribution of outcomes from a probabilistic syst ... Statistics & Methodology Multi-Touch Attribution An attribution framework that distributes conversion credit across multiple marketing touchpoints in ... Analytics & Attribution Multiple Comparisons Problem The increased probability of falsely identifying a significant result when conducting multiple simul ... Statistics & Methodology Multivariate Testing An experiment that tests multiple variables simultaneously to find the optimal combination, rather t ... A/B Testing

N

Net Revenue Retention (NRR) The percentage of recurring revenue retained from an existing customer cohort over a time period, in ... Analytics & Attribution Network Effects A phenomenon where a product becomes more valuable to each user as more users join — the foundationa ... Analytics & Attribution Network Effects in Experiments Interference between experiment groups that occurs when one user's treatment affects another user's ... A/B Testing Normal Distribution A symmetric, bell-shaped probability distribution defined by its mean and standard deviation, where ... Statistics & Methodology North Star Metric The single metric that best captures the core value your product delivers to customers, and that the ... Analytics & Attribution North Star Metric (Experimentation) The single metric that best captures the core value your product delivers to customers — used to ali ... Experimentation Strategy Novelty Effect (in Testing) A temporary lift in engagement or conversion caused by the newness of a change rather than its inher ... A/B Testing Nudge (Libertarian Paternalism) An approach to choice architecture that steers people toward better decisions while preserving their ... Behavioral Economics Nudge Theory A concept in behavioral science that proposes designing choice environments to influence decisions w ... Behavioral Economics

O

Omission Bias The tendency to judge harmful actions as worse than equally harmful inactions, leading people to pre ... Behavioral Economics Omniconvert A conversion optimization platform combining A/B testing, on-site surveys, and customer segmentation ... Experimentation Strategy Optimism Bias The tendency to overestimate the likelihood of positive events and underestimate the likelihood of n ... Behavioral Economics Optimizely An enterprise experimentation platform offering A/B testing, feature management, and personalization ... Experimentation Strategy

P

Page Speed & Conversion The relationship between how quickly a web page loads and renders and its conversion rate, where fas ... Conversion Rate Optimization Pain of Paying The negative emotional response triggered by spending money, which varies in intensity depending on ... Behavioral Economics Paradox of Choice The counterintuitive finding that having too many options leads to decision paralysis, lower satisfa ... Behavioral Economics Paradox of Choice (Psychology) The psychological phenomenon where an abundance of options, rather than liberating, causes anxiety, ... Behavioral Economics Paradox of the Active User The observation that users prefer to start using a system immediately rather than reading instructio ... UX & Digital Psychology Payback Period The number of months required for a customer's gross profit contribution to recover the cost of acqu ... Analytics & Attribution Peak-End Rule People judge experiences based on the most intense moment (peak) and the final moment (end), not the ... Behavioral Economics Personalization The practice of dynamically tailoring web content, offers, and experiences to individual users based ... Conversion Rate Optimization Personyze A mid-market personalization and A/B testing platform offering behavioral targeting, recommendations ... Experimentation Strategy Post-Hoc Analysis Analysis performed after an experiment concludes that was not part of the original test plan — usefu ... A/B Testing Posterior Distribution In Bayesian statistics, the updated probability distribution of a parameter after combining prior be ... Statistics & Methodology Power Analysis The pre-experiment calculation that determines the sample size required to detect a given effect wit ... Statistics & Methodology Pre-Registration (Experiment Protocol) The practice of formally documenting an experiment's hypothesis, metrics, sample size, and analysis ... A/B Testing Present Bias The tendency to overvalue immediate rewards and undervalue future rewards, leading to decisions that ... Behavioral Economics Primacy Effect in Testing A bias where existing users resist changes to familiar interfaces, causing a negative initial reacti ... A/B Testing Prior Distribution In Bayesian statistics, the probability distribution representing existing beliefs or knowledge abou ... Statistics & Methodology Product Analytics The discipline of analyzing user behavior inside a digital product — distinct from marketing analyti ... Analytics & Attribution Product Qualified Lead (PQL) A user who has experienced meaningful value in a free or trial product and demonstrated behaviors su ... Analytics & Attribution Product-Led Growth (PLG) A go-to-market strategy where the product itself drives acquisition, activation, retention, and expa ... Analytics & Attribution Progressive Disclosure A design strategy that shows only the most essential information and options initially, revealing ad ... UX & Digital Psychology Propensity Score Matching A quasi-experimental technique that matches treated and untreated subjects based on their predicted ... Analytics & Attribution Prospect Theory A behavioral economics theory that describes how people evaluate potential gains and losses asymmetr ... Behavioral Economics

R

Rage Clicks A pattern of rapid, repeated clicks on the same element or area, indicating user frustration caused ... Conversion Rate Optimization Randomization Unit The entity (user, session, pageview, device, or account) that is randomly assigned to a test variant ... A/B Testing Rapid Prototyping for Experiments The practice of quickly building low-fidelity test variants to validate concepts before investing in ... Experimentation Strategy Reactance The psychological resistance that arises when people feel their freedom of choice is being threatene ... Behavioral Economics Reciprocity Principle The social norm that compels people to return favors — when you give something of value first, peopl ... Behavioral Economics Recognition Over Recall The cognitive principle that people are better at recognizing previously encountered information tha ... UX & Digital Psychology Redirect Testing An A/B testing method where visitors are redirected to entirely different URLs rather than seeing va ... A/B Testing Regression Discontinuity A quasi-experimental design that identifies causal effects by comparing units just above and just be ... Statistics & Methodology Regret Aversion The tendency to avoid making decisions that might lead to regret, often resulting in inaction, risk ... Behavioral Economics Reinforcement Learning for Experimentation Using RL algorithms — multi-armed bandits, contextual bandits, and full RL — to adaptively allocate ... Statistics & Methodology Representativeness Heuristic A cognitive shortcut where people judge the probability of something by how closely it matches a pro ... Behavioral Economics Retention Curve A graph showing the percentage of a user cohort still active at each time period after signup — the ... Analytics & Attribution Return on Ad Spend (ROAS) The revenue generated for every dollar spent on advertising, calculated as (revenue from ads / ad sp ... Conversion Rate Optimization Revenue Attribution The practice of connecting specific revenue outcomes to the marketing activities, channels, and touc ... Analytics & Attribution Revenue Churn vs Logo Churn Two distinct churn metrics: logo churn measures the percentage of customers lost; revenue churn meas ... Analytics & Attribution Risk Appetite in Experimentation An organization's tolerance for potential negative outcomes from experiments, which determines what ... Experimentation Strategy Roll-Out Strategy The planned approach for gradually deploying a winning A/B test variant to the full user population, ... A/B Testing

S

Sales-Led Growth (SLG) A go-to-market strategy where a human sales team is the primary driver of revenue, typically require ... Analytics & Attribution Sample Ratio Mismatch (SRM) A diagnostic check that detects when the actual traffic split in an A/B test differs significantly f ... A/B Testing Satisficing A decision-making strategy where people choose the first option that meets their minimum acceptable ... Behavioral Economics Scarcity Bias The tendency to place higher value on things that are scarce or perceived as limited in availability ... Behavioral Economics Scroll Depth A measurement of how far down a page users scroll, typically expressed as a percentage of total page ... Conversion Rate Optimization Segment (CDP for Experimentation) A Customer Data Platform (now part of Twilio) that collects, unifies, and routes user event data to ... Experimentation Strategy Segmentation in A/B Tests The practice of analyzing A/B test results across different user subgroups to understand whether tre ... A/B Testing Selection Bias A systematic error that occurs when the sample studied is not representative of the population inten ... Statistics & Methodology Sequential Testing A statistical methodology for monitoring A/B test results as data accumulates in real time, using ad ... A/B Testing Serial Position Effect The tendency to remember items at the beginning (primacy effect) and end (recency effect) of a list ... Behavioral Economics Server-Side Testing An experimentation approach where variations are rendered on the server before the page is delivered ... A/B Testing Session Recordings Playback videos of individual user sessions that capture mouse movements, clicks, scrolls, and page ... Conversion Rate Optimization Signifier A perceivable indicator that communicates what action is possible or how an element should be used, ... UX & Digital Psychology Social Proof The psychological tendency to look to others' actions and opinions when deciding what to do — especi ... Behavioral Economics Split URL Testing An experimentation method that tests entirely different page URLs or designs against each other by s ... A/B Testing Split.io A feature delivery and experimentation platform focused on engineering teams, with strong support fo ... Experimentation Strategy Standard Deviation A measure of the average amount of variability in a dataset, representing how spread out values are ... Statistics & Methodology Statistical Power The probability that a test will correctly detect a true effect when one exists — conventionally set ... Statistics & Methodology Statsig A modern experimentation and feature management platform with strong data-warehouse integration, a g ... Experimentation Strategy Status Quo Bias The preference for the current state of affairs, where any change from the baseline is perceived as ... Behavioral Economics Stratified Sampling A sampling method that divides a population into distinct subgroups (strata) and samples from each p ... Statistics & Methodology Sunk Cost Fallacy The tendency to continue investing in something because of previously invested resources (time, mone ... Behavioral Economics Survivorship Bias The logical error of focusing on entities that passed a selection process while overlooking those th ... Statistics & Methodology Synthetic Control Method A causal inference technique that constructs an artificial control group by weighting untreated unit ... Analytics & Attribution System 1 vs System 2 Thinking The dual-process theory of cognition where System 1 operates fast, automatic, and intuitive, while S ... Behavioral Economics

T

T-Test A statistical hypothesis test that compares the means of two groups to determine if they are signifi ... Statistics & Methodology Tealium iQ An enterprise tag management system (TMS) commonly used to deploy experimentation, analytics, and ma ... Experimentation Strategy Test & Learn Culture An organizational mindset where decisions are validated through experiments rather than authority, a ... Experimentation Strategy Test Democratization The practice of enabling non-specialist teams across the organization to design, launch, and analyze ... Experimentation Strategy Test Duration The length of time an A/B test should run to produce valid results — balancing the need for sufficie ... A/B Testing Test Stopping Rules Pre-defined criteria that specify when an A/B test should be concluded, preventing both premature st ... A/B Testing Test Velocity The rate at which an organization launches, concludes, and learns from experiments — a key indicator ... A/B Testing Thompson Sampling A Bayesian bandit algorithm that allocates traffic to variants based on the probability that each va ... A/B Testing Thumb Zone Design The practice of placing interactive elements within the area of a mobile screen that is comfortably ... UX & Digital Psychology Time to Value (TTV) The elapsed time from a user's first interaction with a product until they experience its core value ... Analytics & Attribution Type I Error (False Positive) Concluding that a variant beat control when in reality there is no true effect — the risk is control ... Statistics & Methodology Type II Error (False Negative) Failing to detect a real effect — the risk is controlled by statistical power and equals 1 minus pow ... Statistics & Methodology

U

Unbounce A landing page builder with integrated A/B testing, focused on marketers running paid traffic campai ... Experimentation Strategy Uplift Modeling A machine learning approach that predicts the causal effect of a treatment on an individual user — w ... Statistics & Methodology User Journey Mapping The process of visualizing the complete end-to-end experience a user has with a product or service, ... Conversion Rate Optimization UTM Parameters Standardized URL query parameters (source, medium, campaign, term, content) appended to links to tra ... Analytics & Attribution

V

Value Proposition Testing The systematic process of testing different articulations of your product or service value against e ... Conversion Rate Optimization Variance The average of the squared differences between each data point and the mean, measuring the spread of ... Statistics & Methodology Variance Reduction Techniques Methods that shrink the noise in experiment metrics — including CUPED, stratification, post-stratifi ... Statistics & Methodology Variant Design The process of creating the experimental treatment in an A/B test, translating a behavioral hypothes ... A/B Testing Viral Coefficient (K-Factor) The average number of new users a single existing user generates through referrals or sharing — a K- ... Analytics & Attribution Visual Hierarchy The arrangement of design elements in order of importance using size, color, contrast, spacing, and ... UX & Digital Psychology Von Restorff Effect (Isolation Effect) The tendency for an item that stands out as distinctly different from its surroundings to be more ea ... Behavioral Economics VWO (Visual Website Optimizer) A CRO-focused experimentation platform that combines A/B testing with behavioral analytics tools lik ... Experimentation Strategy

W

Webtrends Optimize A UK-based experimentation and personalization platform offering A/B testing, server-side testing, a ... Experimentation Strategy Winner Determination The process of deciding which variant in an A/B test has genuinely outperformed the control, using s ... A/B Testing

Z

Z-Pattern Layout A visual scanning pattern where the eye moves in a Z-shape across a page: left to right across the t ... UX & Digital Psychology Z-Score A standardized score indicating how many standard deviations a data point or test statistic is from ... Statistics & Methodology Zeigarnik Effect The psychological tendency to remember incomplete or interrupted tasks better than completed ones, c ... Behavioral Economics Zero Risk Bias The preference to completely eliminate one risk rather than achieving a larger overall reduction in ... Behavioral Economics