The Unmeasured Cost of Bad UX: What Your Funnel Won't Show You

When the Federal Trade Commission filed against Amazon in 2023, one detail stuck with me. According to the complaint, Amazon's own engineers had a nickname for the Prime cancellation flow. They called it the Iliad Flow — after Homer's epic, because the journey to cancel was that long. Six pages. Fifteen clicks. Multiple confirmation prompts dressed up as deals. Internal documents allegedly showed it was deliberate.

If you only looked at Amazon's dashboard, the Iliad Flow looked like a triumph. Cancellation rates were down. Retention curves were up. The funnel team had every reason to celebrate.

But the cost was real. It just wasn't on the dashboard.

This is the problem I want to talk about: the gap between what your analytics can see and what is actually happening to your business. Steve Krug's _Don't Make Me Think_ taught a generation of designers to obsess over usability friction. What it could not have taught — because the regulatory and economic landscape has shifted so far in the eleven years since the third edition — is that the cost of getting usability wrong now compounds in places dashboards were never designed to look.

Your funnel measures who showed up. It cannot measure who never came back, who told their coworkers to stay away, or who triggered a class action against you.

I've been running experiments and consulting on conversion programs for over a decade. The pattern I see most often is not that companies have bad UX. It is that companies have UX whose true cost is structurally invisible to the metrics they trust. They are flying instruments-only through weather their instruments cannot detect.

This essay is about how to see what you are missing — and the framework I use with clients to find it.

The dashboard tells you the funnel. The funnel does not tell you the business.

Here is the first thing I tell every CMO who hires me to look at their conversion program: your funnel drop is a symptom, not a diagnosis. And it is not even all the symptoms.

A standard analytics stack — GA4, Amplitude, Mixpanel, your warehouse, your BI tool — is built to measure one thing well: the journey of users who already showed up, through the steps you already designed, toward the events you already chose to track. It is a survivorship-biased instrument by construction.

When I led growth at a B2B SaaS company a few years back, our funnel told a clean story. Signup-to-activation was up. Activation-to-paid was up. Net revenue retention was up. We were a competence machine. Then a researcher showed me the cancellation free-text data — what people actually wrote when they churned. About one in three citations talked not about price or feature gaps but about how it felt to use the product. _Talked down to. Forced through tutorials. Couldn't find the cancel button when I needed it. Won't recommend._ None of that was anywhere on the funnel dashboard.

The funnel dashboard was honest. It just was not complete.

Krug's First Law of Usability is "Don't make me think." Every single principle in his book — scanning over reading, mindless choices, omitting needless words, courtesy as design — is a defense against the cumulative cost of small frictions. What Krug could not anticipate, writing in 2014, is that those small frictions now have a price tag set by judges, by regulators, and by the algorithmic word-of-mouth machine we call social media.

The principles still hold. The cost of breaking them has changed.

The four invisible costs

When I do a UX audit for a client, I am not just looking at usability. I am estimating four cost lines that almost never appear in the conversion narrative the company has been telling itself.

1. Word-of-mouth

When a user has a bad cancellation experience, they do not just churn. They tell people. They complain to coworkers. They post in Slack groups. They warn friends.

This is the part of Bain & Company's Net Promoter research that gets quoted least often: detractors talk more than promoters. The asymmetry is real. A frustrated user who could not find the cancel button generates outsized negative referrals — not in the quarter they cancelled, but in the months and years afterward, every time someone in their network considers your category and remembers what they were told.

You do not see this on your acquisition dashboard. You see it as a slow, untraceable degradation in cost-per-acquisition. Your paid channels work harder for the same number of leads. Your organic traffic does not compound the way it should given your content investment. The marketing team blames the algorithm. It is not the algorithm. It is the customers you angered last year.

2. Legal exposure

Eleven years ago, a clumsy cancellation flow was a usability problem. Today it is a federal offense.

The FTC's Click-to-Cancel rule, finalized in late 2024, requires that cancelling a subscription be at least as easy as starting one. The Amazon case I opened with settled in September 2025 for $2.5 billion total — $1 billion civil penalty plus $1.5 billion in consumer refunds, the largest ROSCA settlement on record. State-level enforcement — California's Consumer Privacy Act regulations explicitly prohibit dark patterns — is layered on top. The European Union's Digital Services Act, fully applicable as of 2024, names dark patterns directly in Article 25 with revenue-scaled penalties.

This is the part most CFOs miss when they ask me whether usability is "really" a priority. Usability is now a compliance line item. A bad cookie banner is no longer a friction inconvenience — it is litigation surface area. Plaintiffs' law firms have noticed. They are systematically targeting small and mid-sized businesses for cookie consent and accessibility violations because those companies tend to settle quickly. The FTC's $245 million settlement against Epic Games in 2022 — for billing dark patterns inside Fortnite — was the warning shot. The Amazon settlement two years later is the answer to whether the warning was heard.

If your audit posture is "we have not been sued yet," you are a risk pool, not a strategy.

3. Employee trust collapse

I have seen this play out more times than I would like. A growth team ships a conversion-optimized cancellation flow. The metrics look good. But two months later, the engineers who built it stop volunteering for retention work. The customer-success team starts using qualifying language when they explain the flow to new hires. The internal Slack channel where people post their company's reviews on G2 goes quiet.

You are not just losing customers. You are losing the internal advocacy that quietly sustains every successful product. Employees who do not believe in what they shipped do not refer friends, do not stay as long, and do not bring their best ideas. Decades of workplace research — Gallup's State of the Workplace series, Glassdoor's culture studies — point to the same finding: pride in the product is among the strongest predictors of retention and discretionary effort.

This cost is the hardest to measure and the most expensive to recover from. Once your team has lost faith in the user experience, they bring that loss to every subsequent design decision.

4. Brand half-life

The economic frame I rely on most when I make this case to a CEO comes from Les Binet and Peter Field's work on the IPA Databank. Their long-running analysis of marketing effectiveness data found that brand-building activities show payback over multi-year horizons — typically two to three years or longer — while performance marketing shows payback in weeks. The corollary they did not state but that is implicit in their data: brand damage from a negative customer experience compounds on the same multi-year clock.

This means the bad UX you shipped this quarter does not show up in your business case until somewhere between month twelve and month twenty-four. By the time the cost is visible, the team that shipped it is gone, the dashboard owner has changed, and the line item is invisible inside a "softening market" or "category headwinds" narrative. Nobody connects the present-quarter pain to the past-year design choice.

I call this the brand half-life. The decay is real, predictable, and almost completely uncounted.

SHADOW check: This is the proxy logic I run with every client. If you are auditing a UX change, watch six signals your funnel does not surface — Sentiment trends, Help-desk volume by reason, Audits and legal exposure, Defection reasons in cancellation free-text, Outside reviews thematically analyzed, and Word-of-mouth via NPS detractor counts. Together they form what I call SHADOW. The framework is below.

Why your dashboard is structurally blind

Once you accept that those four costs are real, the next question is: why don't I see them?

The answer is that your dashboard was not designed to. It was designed to optimize the funnel you have, not to detect the funnel you might have damaged.

Three structural blind spots compound:

Survivorship bias. Every event in your analytics stack comes from a user who showed up. The user who heard about your bad cancellation experience from a coworker and decided not to try you in the first place is, by definition, absent from your data. The acquisition cost they would have represented at zero friction is invisible. You see who came; you cannot see who was warned away.

Lag. Brand effects compound over months and years. Your dashboard reports daily, weekly, quarterly. The mismatch between the timescale of cause and the timescale of measurement guarantees that by the time the harm is statistically visible, the diagnostic trail to the original UX choice has gone cold.

Channel attribution. Your acquisition team measures by channel — paid, organic, referral, direct. None of those channels capture the conversation that happened on Slack, in a Discord, at a dinner. Word-of-mouth is the dominant channel for many products, and it is the only channel without instrumentation. When that channel degrades, every other channel has to work harder, but no one can name what changed.

Most growth teams I work with have built thoughtful, sophisticated funnel dashboards. The dashboards are not wrong. They are just incapable of catching the costs that show up next year.

I tell clients: you cannot test what you cannot measure, and you cannot measure what you cannot see. Step one is admitting where your instruments are blind.

The SHADOW framework

This is the audit I run with clients to surface the costs the dashboard does not catch. SHADOW is six proxies. It is not a replacement for your funnel — it is the second instrument panel that hangs alongside it.

A few notes on how to use it.

Sentiment is not your social listening tool's overall score. It is the trend over time on specific topics. When was the last time someone wrote about your cancellation flow? When was the time before that? Are the topics shifting from product-feature mentions to friction mentions?

Help-desk volume by reason is the most under-mined data source in most companies. Your support team already tags tickets. Group them by the UX pattern they trace back to — "where is the cancel button," "couldn't change my plan," "the form didn't accept my address" — not just by topic. The cluster sizes tell you which Krug principles you are violating in the wild.

Audits is the leg most CMOs forget. Run a quarterly check on three things: are your cancellation flows compliant with the Click-to-Cancel rule, are your cookie consent mechanics defensible against an EU regulator's read of the DSA, and are your accessibility audits current against WCAG 2.2. Each of these is now a six-to-seven-figure liability if neglected.

Defection reasons is a goldmine and is almost never read. Cancellation free-text — the box people fill in when they leave — contains, in plain language, the costs that did not appear in your funnel. Read it monthly. Code the themes. Watch which themes grow.

Outside reviews are not a stars game. The stars are noise. The themes are signal. Are people praising or complaining about specific experiences? Is the language about your onboarding shifting?

Word-of-mouth is the proxy with the heaviest research lineage. Use Bain's Net Promoter framework — but use the detractor count, not just the score. Multiply by the category-appropriate negative-WOM coefficient. Treat the result as the lost-acquisition figure that does not appear on your funnel.

{{ShadowDownloadCTA}}

I built SHADOW originally to give CMOs a vocabulary their CFOs could engage with. The funnel reports a number; SHADOW reports a portfolio of risk. When I present a UX recommendation now, I do not just show conversion delta. I show SHADOW deltas. That has changed which projects get funded.

What this means for CRO

I want to close with the practical implication, because it changes how I run conversion programs and I think it should change how you run yours.

If you optimize only for funnel metrics, you will systematically over-invest in extraction patterns — the small UX choices that shave a percent off cancellation rate, push another user through onboarding, force one more upgrade modal — and systematically under-invest in courtesy. You will hit your quarterly numbers. You will erode the business that produces those numbers.

This is the part of Krug's principle on courtesy as design that I think the field has under-read. Courtesy is not aesthetic. It is the only sustainable conversion strategy. Every pattern that violates it is borrowing from the brand half-life and pretending the loan does not exist.

Test in your context. The decoy effect, anchoring, multi-step forms, urgency badges — all of them work in some environments and not others. The standard CRO playbook treats these as universal. They are not. They are heuristics that interact with your competitive position, your audience age, your category maturity, and your audience's prior experience with similar patterns. Anchoring breaks if your product is undifferentiated. Multi-step forms break if motivation is low. The honest answer is always: run the test in your environment, with your audience, and read the SHADOW proxies alongside the funnel.

Don't trust blindly. Test, watch the funnel, watch SHADOW, then decide.

The insight I keep coming back to with clients: just because it is not showing up on your dashboard does not mean it is not happening. You might not be measuring it. You might not have the capability to measure it. But the effects are real, and they will be visible — eventually — in places that hurt more than the funnel.

The cost is being paid. The only question is whether you can see the bill yet.

FAQ

What is the difference between a UX problem and a dark pattern?

A UX problem is unintentional friction — a button placed badly, a confusing label, an unclear next step. A dark pattern is intentional friction — a design choice deliberately structured to trick users into outcomes that benefit the business at their expense. Krug's principles address UX problems. Princeton's Dark Patterns at Scale study (Mathur et al., 2019) and the regulatory landscape since 2022 have shifted dark patterns from a UX critique into a compliance category. A bad cancellation flow that confuses users by accident is a usability bug. A cancellation flow with an internal nickname like "Iliad" is a regulatory liability.

How is SHADOW different from a standard CX audit?

A standard customer experience audit measures satisfaction at a moment in time — usually via NPS or a CSAT survey. SHADOW is built around the problem most CX programs have: the costs of bad UX show up months or years later, in places your survey instrument cannot see. SHADOW deliberately weights leading indicators (sentiment trends, help-desk theme shifts, defection free-text) and second-order indicators (regulatory exposure, word-of-mouth degradation) over point-in-time satisfaction scores. It is designed to catch the four invisible costs before they become visible costs.

Should I stop using funnel analytics?

No. Funnel analytics are essential — they tell you what is happening to the users who showed up. SHADOW does not replace that instrument; it is the second one. The point is to read both panels at once. When SHADOW shows degradation that the funnel does not, that is the early signal that you are extracting from the brand half-life. When the funnel shows degradation that SHADOW does not, you have a tactical conversion problem. When both show degradation, you have a product problem.

How often should I run a SHADOW audit?

Quarterly is the minimum I recommend for any company over 100 employees. The proxies move on a slower clock than funnel metrics, so weekly is excessive — you'll see noise. The exception is the Audits leg: legal and accessibility exposure should be checked any time you ship a material change to a checkout, signup, or cancellation flow, because new regulatory enforcement makes the same pattern more expensive over time.

What does a SHADOW score actually look like?

I score each proxy on a 1–5 scale based on directional movement and current state, for a total out of 30. Above 25 is healthy; the funnel and the brand are aligned. 15–24 is "surface concerns" — typically one or two proxies are degrading and you are eroding the brand half-life without seeing it on the funnel yet. Below 15 is active brand risk and usually correlates with cancellation patterns that need legal review. The downloadable framework walks through how to score each proxy.

Run a SHADOW audit on your own program

If your funnel looks healthy but you have a quiet sense the business is not, that is the signal SHADOW is built to read. The framework is a one-page audit you can run on your own program in an afternoon.

Or if you want a second pair of eyes — I work with a small number of growth teams every quarter to do this audit alongside their CRO program. Book a strategy call and we will look at your SHADOW proxies together.

Either way: the cost your dashboard does not see is the cost that compounds. The first job is to see it.