Hypothesis Testing
The practice of formulating a specific, falsifiable prediction before running an experiment — the foundation of scientific experimentation.
A hypothesis in experimentation is a specific, testable prediction about what will happen and why. Without one, you're not experimenting — you're guessing with extra steps.
The Hypothesis Framework
A strong experimentation hypothesis follows this structure:
"We believe that [specific change] will cause [measurable outcome] for [target audience] because [behavioral or data-driven rationale]."
Example: "We believe that adding a progress bar to the checkout flow will increase completion rate by 5-10% for mobile users because the Zeigarnik Effect creates psychological momentum to finish started tasks."
Why Hypotheses Matter
- Focus: Forces you to identify the primary metric before you start
- Learning: A rejected hypothesis teaches you something specific about user behavior
- Accountability: Prevents post-hoc rationalization ("we were really testing X all along")
- Prioritization: Stronger rationale = higher confidence = higher priority
The Hypothesis Library
The most mature experimentation programs maintain a hypothesis library — a documented collection of tested hypotheses, results, and learnings. This becomes the organization's behavioral science knowledge base, preventing repeated mistakes and accelerating future test design.
Common Hypothesis Mistakes
- Too vague: "We believe a new design will improve conversion" (no mechanism, no rationale)
- Too specific: "We believe changing the button from blue to green will increase clicks by exactly 7.3%"
- No behavioral rationale: "We believe this will work because our designer thinks it looks better"