Mann-Whitney U Test
A non-parametric statistical test that compares two independent groups without assuming normal distribution, testing whether one group tends to have larger values than the other.
What Is the Mann-Whitney U Test?
The Mann-Whitney U test compares two independent groups using ranks. It does not require normality, but it still assumes independent observations and tests a rank or distributional difference. Interpreting it as a median test requires additional shape assumptions.
Also Known As
- Data science teams: Mann-Whitney-Wilcoxon, Wilcoxon rank-sum, non-parametric t
- Growth teams: robust significance test
- Marketing teams: the test you use when revenue data is weird
- Engineering teams: rank test, U-test
How It Works
Imagine an A/B test with 10,000 visitors per variant measuring revenue-per-visitor. Most users contribute $0, a handful contribute hundreds. A t-test on this data is dragged by outliers and may be unreliable. Instead, you rank all 20,000 values from smallest to largest and sum the ranks in each group. If the ranks in Variant B are systematically higher, the U-statistic will be large, and the p-value will be small. This sidesteps the outlier problem because the top buyer just gets rank 20,000 regardless of whether they spent $500 or $50,000.
Best Practices
- Do use Mann-Whitney when its rank-based estimand matches the business question and the independence assumptions hold.
- Do confirm your tooling handles ties correctly; ties can inflate false positive rates if mishandled.
- Do pair the U-statistic with a median difference or Hodges-Lehmann estimator for business context.
- Do not use Mann-Whitney when your hypothesis is specifically about the mean.
- Do not assume Mann-Whitney tests equality of distributions; it tests stochastic dominance.
Common Mistakes
- Reporting Mann-Whitney p-values without any measure of effect size or direction.
- Forgetting that with very large samples, Mann-Whitney will flag trivially small ordering differences.
- Confusing the U-statistic with a z-score; modern implementations convert it internally.
Industry Context
- Rank-based tests may be useful across industries when the estimand is an ordering or distributional shift. They do not answer a mean-revenue question merely because the data is skewed.
The Behavioral Science Connection
Rank-based reasoning reflects how humans often process comparisons in the real world: we know a restaurant is better than another without knowing exact ratings. Thaler's work on relative valuation shows people are more comfortable ranking than quantifying. Mann-Whitney respects that intuition by only asking "is A usually greater than B?"
Key Takeaway
Mann-Whitney is a rank-based test, not an assumption-free default for skewed revenue. Choose it only when its estimand matches the decision.