Propensity Score Matching

What Is Propensity Score Matching?

Propensity score matching (PSM) is a statistical technique for estimating treatment effects when randomization is impossible. Developed by Rosenbaum and Rubin (1983), PSM calculates each subject's propensity (probability) to receive treatment based on observed characteristics, then matches treated subjects with untreated subjects of similar propensity to create a comparable counterfactual.

Also Known As

• Marketing team: "matched-cohort analysis"
• Sales team: "look-alike comparison"
• Growth team: "propensity matching," "PSM"
• Data team: "propensity score analysis," "matched pairs design"
• Finance team: "comparable-cohort ROI"
• Product team: "adopter vs. non-adopter matched study"

How It Works

You want to measure the lifetime value impact of your loyalty program. You can't randomize — customers self-select. Step 1: predict each customer's probability of joining based on pre-period behavior (purchase history, engagement, session count). Step 2: for each program member, find a non-member with a similar propensity score. Step 3: compare LTV between matched pairs. If matched program members have $800 LTV and matched non-members have $650, the estimated program lift is $150 — with selection bias partially removed.

Best Practices

• Include every observable variable that might drive both treatment selection and the outcome.
• Check covariate balance after matching — if matched groups still differ, the match failed.
• Report results with and without matching to show the bias correction effect.
• Use sensitivity analysis to estimate how much hidden bias would be needed to overturn your conclusion.
• Treat PSM results as evidence, not proof — always weaker than a randomized test.

Common Mistakes

• Assuming PSM controls for unmeasured confounders (it doesn't — motivation, intent, information access remain invisible).
• Including post-treatment variables in the propensity score, which biases the estimate.
• Reporting PSM results as if they were RCT results.

Industry Context

SaaS and B2B teams use PSM for features that can't be randomized — voluntary opt-in features, premium tier adoption, customer advocacy programs. Ecommerce and DTC teams use PSM for loyalty programs, subscribe-and-save evaluation, and retroactive campaign analysis. Lead gen operations apply PSM when evaluating programs that rolled out before proper test design was in place.

The Behavioral Science Connection

Self-selection is itself a behavioral phenomenon. People who opt into loyalty programs, premium tiers, or beta features exhibit commitment bias and loss aversion — they want to lock in benefits. This systematic difference between joiners and non-joiners is exactly what PSM tries to control for. Understanding the behavioral drivers of selection helps you choose better matching variables.

Key Takeaway

PSM is the best answer when the best method (randomization) isn't available — but always report its limitations alongside its results.