Silence is data

A product rarely dies of something a user told you. It dies of the things they didn't.

We wanted to know why some early products survive their first real contact with a market and others quietly do not. The pattern in the research is not about effort or talent. It's about signal — specifically, the signal that never arrives.

What the research says

When CB Insights read through a long list of startup post-mortems, the single most common cause of death was not running out of money or being out-built. It was making something the market didn't want — roughly a third of the failures. None of those teams failed to ship. What they lacked was the signal that they were shipping the wrong thing, early enough to act on it. Steve Blank's case for the lean startup, in Harvard Business Review, makes the same point structurally: the companies that live are the ones whose loop from build to feedback is short enough to catch the error while it's still cheap. The failure is not making the mistake. It is making it for a year.

The reason the signal is hard to get is older than software. The classic finding in customer research is that for every customer who complains, around twenty-five say nothing at all. The unhappy user is not rare. They're quiet. The channels that feel like listening — support tickets, the occasional reply to an email — sample the loud and the still-present, and structurally miss the user who is already halfway out the door and has stopped bothering to say why.

That quiet group is exactly the one the retention math is about. Summarizing decades of work by Frederick Reichheld, Amy Gallo notes in Harvard Business Review that keeping an existing customer costs a fraction of winning a new one, and that small movements in retention compound into large ones. Every silent exit is a customer you were never given the chance to keep — the cost lands later, on a dashboard, with no name attached to it.

Our take

The research is usually read as a volume problem: collect more feedback, run more surveys, raise the response rate. It's a selection problem — survivorship bias.

In 1943, Abraham Wald was asked where to add armor to bombers, working from the bullet holes in the planes that came back from missions. The intuitive answer was to reinforce the most-hit areas. Wald pointed at the parts with no holes instead: a plane hit there was a plane that didn't return, so its damage was never in the data. The holes on the survivors marked the places a bomber could be shot and still make it home.

Feedback works the same way. The users who would tell you the most are the planes that did not come back — they have already half-left, and they were never going to file a ticket. The survey responses you do collect are the bombers that landed: evidence of where a user can be hit and still stay. A bigger survey just inspects more returning planes.

So the work isn't reading the survivors more closely. It is going to the quiet user, in the one place they still are — inside the product, mid-task, in the few seconds before the tab closes — and asking while there's still someone there to answer.

What we do

We put the ask where the half-leaving user still is: in the product, not in an inbox they have stopped opening. We keep it to one question, and we put a real name and face on it, because a request from a person gets answered where a request from a system gets dismissed. A short check-in a few days after signup reaches the user who hit friction while they are still around to describe it, instead of counting them later as one of the twenty-five.

The question was never whether your users would tell you. It's whether you asked while they were still there to answer.

Sources

• CB Insights. The Top Reasons Startups Fail.
• Blank, S. (2013). Why the Lean Start-Up Changes Everything. Harvard Business Review.
• Gallo, A. (2014). The Value of Keeping the Right Customers. Harvard Business Review.
• Lee Resources / TARP — the classic finding that for every customer who complains, roughly twenty-five stay silent; accessible write-up via Groove.
• Wald, A. (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Statistical Research Group, Columbia University; recounted in Ellenberg, J. (2014). How Not to Be Wrong.