Full Factorial Design

What Is Full Factorial Design?

Full factorial design is a multivariate experimental structure that tests every combination of the factors and levels under test. Three headlines × four button colors = twelve cells, each tested independently. It's the only approach that reliably reveals interaction effects — cases where specific combinations perform differently than the sum of their individual parts — at the cost of multiplicatively more traffic than a simple A/B test.

Also Known As

• Marketing teams call it MVT or combination test.
• Growth teams say full factorial, MVT, or combinatorial test.
• Product teams use factorial design or multivariate test.
• Engineering teams refer to full factorial or 2^k design.
• Statisticians strictly call it full factorial design.

How It Works

You test 2 headlines × 2 images × 2 CTAs = 8 combinations. Each needs adequate sample size — say 10,000 visitors per cell for 80% power. Total: 80,000 visitors. Results: headline A wins on its own (+3%), image 1 wins on its own (+2%), CTA X wins on its own (+1%). Simple addition predicts the A+1+X combination should win by 6%. Actual result: A+1+X wins by 9%. The extra 3% is interaction — A and 1 amplify each other because they share a tone and aesthetic. Only full factorial reveals this.

Best Practices

• Use full factorial when you have 2–3 factors with 2–3 levels each — keep total cells under 12.
• Calculate cell-level sample size requirements up front; underpowered cells invalidate the whole test.
• Analyze main effects first, then two-way interactions, then higher-order interactions (if any).
• Pre-register which interactions you care about before seeing results.
• Use full factorial only when traffic allows — otherwise use fractional factorial.

Common Mistakes

• Running full factorial on insufficient traffic, leaving cells with 500 visitors and no statistical power.
• Treating each combination as an independent A/B test and ignoring factorial structure.
• Declaring "winning combinations" from post-hoc slicing without adjusting for multiple comparisons.

Industry Context

• SaaS/B2B: Usually impractical — traffic is too low to support 8+ cells.
• Ecommerce/DTC: High-traffic PDPs can support factorial tests on pricing presentation and merchandising.
• Lead gen: Occasionally useful on high-traffic landing pages for testing copy × image × CTA combinations.

The Behavioral Science Connection

Full factorial reveals coherence effects — where combinations of elements produce meaning neither element creates alone. Confident copy paired with a trustworthy image produces cognitive ease that either alone cannot. It's the only testing structure that exposes these gestalt-level behavioral patterns.

Key Takeaway

Full factorial is the only way to measure interaction effects — use it when traffic supports it and sequential A/B tests would miss the coherence story.