A new CTA producing a small positive topline estimate is not automatically a real win. Some of the movement may be clicks redirected from higher-converting positions on the same page.
TL;DR
- Adding a new CTA to a high-traffic surface can produce a small positive headline estimate.
- That estimate may represent net-new conversions, redistributed clicks, or both. The placement-level breakdown distinguishes them.
- The diagnostic is per-CTA click distribution: pull click volume and click-to-conversion rate for every CTA on the page, in both arms.
- Cannibalization signature: existing CTAs lose click volume in the variant; new CTA has sub-baseline click-to-conversion ratio; total conversions roughly flat.
- Adding three columns to your test report catches it. Skipping the diagnostic is how programs degrade their funnel composition for years.
The two outcomes look identical on the topline
| Outcome | Headline metric | Total page clicks | Conversion efficiency | Decision |
|---|---|---|---|---|
| Real win (additive engagement) | Small positive | Up | New CTA matches or beats existing-CTA click-to-conv | Ship |
| Cannibalized "win" (redistributive) | Small positive | Up | New CTA has sub-baseline click-to-conv; existing CTAs lose click volume | Do not ship |
Both produce a positive topline directionally. The difference shows up only in the per-CTA breakdown. Without the breakdown, the team ships cannibalization disguised as growth.
Illustrative composite: a sitewide nav CTA
This composite combines recurring sitewide-navigation patterns. The values are deliberately expressed as broad bands rather than an attributable experiment or client record.
| Metric | Result |
|---|---|
| Commercial-flow entry | Roughly +1% |
| Downstream completion | Roughly +1% |
| Both within the test's uncertainty | "Looked like" a directional win |
The aggregate said ship. The per-CTA breakdown said the opposite.
| CTA | Clicks (variant arm) | Click-to-next-step rate |
|---|---|---|
| New nav CTA | Several thousand | Mid-single digits |
| Existing intent-matched CTAs | Fewer clicks | Low-to-mid twenties |
| Implication | Total clicks up | New path is several times less efficient |
A large minority of the new CTA's clicks came from account-service pages where audience intent was account access, not evaluation or purchase. The variant was pulling clicks away from intent-matched paths and recapturing some of them through a lower-efficiency modal flow. The roughly one-percent topline estimate did not reveal that composition change.
Decision: do not ship. The CTA stayed off the global nav until copy + routing iterations addressed the source-page intent mismatch.
What the diagnostic catches
The cannibalization signature is consistent. Pull these three numbers per arm and the picture sharpens fast.
| Diagnostic column | Real win signature | Cannibalized "win" signature |
|---|---|---|
| Existing CTAs click volume | Holds steady or grows | Loses click volume in variant |
| New CTA click-to-conversion rate | Matches or exceeds existing CTAs | Sub-baseline (often <half existing) |
| Source-page breakdown | Most clicks from intent-matched pages | Significant share from wrong-intent pages |
If the signatures point at cannibalization, pause the rollout and apply the pre-specified business-outcome rule. A directional topline alone does not resolve whether the funnel composition improved.
Three failure modes the diagnostic catches
| Failure mode | What aggregate shows | What per-CTA breakdown shows |
|---|---|---|
| Wrong-intent clicks | Headline lift on click volume | New CTA captures clicks from source pages where audience intent doesn't match destination |
| Friction injection | Click rate up, conversion flat | New CTA has sub-baseline click-to-conversion ratio because of modal/redirect at destination |
| Cannibalization | Topline lift smaller than expected | Existing CTAs lose click volume; total conversions roughly unchanged |
In every case, the aggregate report is technically correct. The headline number moved on the right side of zero. The team gets a "directional win." The funnel underneath is worse than before the test.
When cannibalization is acceptable
Not every cannibalization signal kills the test. Two patterns where redistribution is genuinely worth it:
| Pattern | Mechanism | When it justifies cannibalization |
|---|---|---|
| Reaching new audience pockets | New CTA pulls clicks from previously-disengaged users | Existing CTAs hold click volume; new CTA adds incremental clicks on top |
| Routing to low-friction destinations | New CTA bypasses a multi-step pre-qualification flow | Per-click revenue lower but conversion rate much higher; total revenue grows |
Both signatures look like "additive engagement," not "redistributive engagement." If the existing CTAs are losing click volume to the new one with no offsetting gain in funnel input, you're in cannibalization territory regardless of what the topline says.
What to add to every CTA test report
Three columns. Each takes one analytics query.
| Column | Question it answers |
|---|---|
| New CTA's click-to-conversion rate | Did the new CTA convert clicks at a healthy rate? |
| Existing CTA click volume per arm | Did the existing CTAs lose volume to the new one? |
| Source-page breakdown of new-CTA clicks | Are clicks coming from intent-matched pages? |
Tests where all three columns look healthy ship. Tests where any column reveals redistribution don't — even if the topline says they do.
The behavioral mechanism
A click is a curiosity event — _what is this_. A conversion is a commitment event — _yes, I want this_. Adding a new CTA to a high-traffic surface raises curiosity events sitewide. That's what visibility does. It does not necessarily raise commitment events at the same rate, because commitment requires:
- Intent alignment between the CTA copy and the destination
- Low friction at the destination
- Content on the destination that matches what the CTA promised
Visibility alone moves only one of the three. Most CTA tests focus on visibility because it's the easiest to spec and the easiest to demo in stakeholder reviews. Cannibalization is the predictable consequence of optimizing visibility without checking whether the new CTA captures incremental commitment or just redistributed curiosity.
Bottom line
A test report that shows aggregate lift without per-CTA composition is incomplete. Pull the per-CTA click distribution and click-to-conversion rate on every CTA test. If the variant arm shows existing CTAs losing click volume to the new CTA at a sub-baseline conversion rate, the win is cannibalized. Do not ship.
The cost of running this diagnostic is one query per test. The cost of skipping it compounds quietly until the funnel is full of CTAs that nobody can defend a unique purpose for.
FAQ
Is a lower click-to-conversion rate proof of cannibalization?
No. It is a diagnostic signal. The new CTA may reach a different audience, serve a different intent, or create incremental volume that still improves total contribution. Compare the control and variant path mix before assigning a cause.
Which metric should decide whether the CTA ships?
Keep the pre-specified business outcome as the decision metric. Use placement-level clicks, next-step completion, and source intent to explain the movement and detect a degraded path mix. Microsoft Research's metric-interpretation guide explains why neat ratios can still mislead when definitions or populations differ.
How long should the downstream window be?
Choose it from the observed decision cycle and state it before reading the result. Google's research on long-term experiment effects shows why a convenient short window may miss delayed outcomes.