Skip to main content

Reference · Glossary

Behavioral Science
& Experimentation
Glossary

Expert-level definitions from a practitioner who's run the experiments. Not textbook theory — applied behavioral science with real A/B test context.

34 Terms Defined

Browse by Category

Behavioral Economics

17 terms

A/B Testing

2 terms

Conversion Rate Optimization

2 terms

Statistics & Methodology

10 terms

Experimentation Strategy

3 terms

A–Z Index

A

A/B Testing A randomized experiment comparing two or more versions of a page or feature to determine which perfo... A/B Testing Anchoring Bias A cognitive bias where people rely too heavily on the first piece of information they encounter when... Behavioral Economics

B

Bayesian A/B Testing An alternative to frequentist A/B testing that uses probability distributions to express uncertainty... Statistics & Methodology

C

Choice Overload The phenomenon where having too many options leads to decision paralysis, reduced satisfaction, and ... Behavioral Economics Cognitive Fluency The ease with which information is processed — fluent (easy) experiences feel more trustworthy, fami... Behavioral Economics Cognitive Load The total amount of mental effort required to process information and make decisions — the invisible... Behavioral Economics Completion Bias The innate human drive to finish what we've started — incomplete tasks create cognitive tension that... Behavioral Economics Confidence Interval A range of values that likely contains the true effect of your A/B test variation — more informative... Statistics & Methodology Conversion Rate Optimization The systematic practice of increasing the percentage of visitors who complete desired actions — usin... Conversion Rate Optimization

D

Decoy Effect A phenomenon where adding a third, inferior option changes preferences between the original two opti... Behavioral Economics Default Effect The powerful tendency for people to accept pre-selected options — making default settings one of the... Behavioral Economics

E

Endowment Effect The tendency for people to overvalue things they already own or feel ownership over, compared to ide... Behavioral Economics Experimentation Culture An organizational mindset where decisions are driven by evidence, hypotheses are tested before shipp... Experimentation Strategy

F

Framing Effect A cognitive bias where people react differently to the same information depending on how it is prese... Behavioral Economics

G

Goal Gradient Effect The tendency for people to increase effort as they get closer to completing a goal — acceleration to... Behavioral Economics

H

Hick's Law The time it takes to make a decision increases logarithmically with the number and complexity of cho... Behavioral Economics Holdout Testing A method for measuring the cumulative impact of all shipped experiments by withholding changes from ... Experimentation Strategy Hypothesis Testing The practice of formulating a specific, falsifiable prediction before running an experiment — the fo... Experimentation Strategy

L

Loss Aversion The psychological tendency for people to prefer avoiding losses over acquiring equivalent gains — lo... Behavioral Economics

M

Minimum Detectable Effect The smallest difference between control and variation that your test is designed to reliably detect ... Statistics & Methodology Multivariate Testing An experiment that tests multiple variables simultaneously to find the optimal combination, rather t... A/B Testing

N

Nudge Theory A concept in behavioral science that proposes designing choice environments to influence decisions w... Behavioral Economics

P

p-value The probability of observing results at least as extreme as the actual results, assuming there is no... Statistics & Methodology Practical Significance Whether an A/B test result is large enough to matter for the business — distinct from statistical si... Statistics & Methodology

R

Revenue Per Session Total revenue divided by total sessions — a more comprehensive optimization metric than conversion r... Conversion Rate Optimization

S

Sample Size The number of visitors or users needed in each variation of an A/B test to detect a meaningful diffe... Statistics & Methodology Satisficing A decision-making strategy where people choose the first option that meets their minimum acceptable ... Behavioral Economics Social Proof The psychological tendency to look to others' actions and opinions when deciding what to do — especi... Behavioral Economics Statistical Power The probability that a test will correctly detect a real effect when one exists — the complement of ... Statistics & Methodology Statistical Significance The probability that an observed difference between test variations is not due to random chance — ty... Statistics & Methodology Status Quo Bias The preference for the current state of affairs, where any change from the baseline is perceived as ... Behavioral Economics Sunk Cost Fallacy The tendency to continue investing in something because of previously invested resources (time, mone... Behavioral Economics

T

Type I Error A false positive — concluding that a test variation is a winner when the observed difference is actu... Statistics & Methodology Type II Error A false negative — concluding that a test variation has no effect when it actually does have a real,... Statistics & Methodology