There is a standard response to product quality problems that most software teams have experienced. Something goes wrong. Users complain. Tickets accumulate. The team investigates, identifies the issue, ships a fix, and closes the tickets. The process works. The problem is solved.
What this process does not reveal is the size of the iceberg beneath the waterline. For every user who filed a ticket, there are users who encountered the same problem and said nothing. They did not write in. They did not leave a review. They just stopped using the product.
Support tickets are evidence that something went wrong. They are not a complete picture of the damage. And by the time they appear, the damage has already happened.
The Iceberg Below the Ticket Queue
Research on customer complaints across different service contexts consistently finds that most users who have a bad experience do not report it. The exact ratio varies by product category, user demographics, and the ease of reporting — but the pattern is consistent: the users who speak up are a minority of the users who were affected.
The implication is that your support queue is a sample of your quality problems, not a census. And it is a biased sample — weighted toward problems that are severe enough to prompt action, from users who are engaged enough to bother. The quiet problems, affecting the less engaged users, are systematically underrepresented.
Lagging vs. Leading Indicators of Quality
In manufacturing and operations, the distinction between lagging and leading indicators is well understood. Lagging indicators tell you what has already happened. Leading indicators give you a signal before the outcome occurs, while there is still time to act.
Most software teams' quality feedback loops are built almost entirely on lagging indicators.
| Signal | What it tells you | Type |
|---|---|---|
| Support tickets | Problems that were severe enough that users who engaged with support reported them | Lagging |
| App store ratings | Extreme experiences — very good or very bad — from users who felt strongly enough to review | Lagging |
| Crash reports | Technical failures that were caught by instrumentation | Lagging |
| Churn and retention metrics | That users left — not why | Lagging |
| Session length and engagement data | Behavioural patterns that may indicate friction or drop-off | Mixed |
| Funnel drop-off analysis | Where users leave a flow — not why | Mixed |
| Pre-launch human evaluation | What real users experience — before it affects your metrics | Leading |
| Structured usability testing | Where users struggle, hesitate, or fail — and why | Leading |
The pattern is clear: the signals most teams rely on most heavily are the ones that tell them what already happened. The signals that could help them act before the damage occurs are the ones most teams underinvest in.
Why This Matters More Than It Used To
The economics of this problem have shifted over the past decade in ways that make lagging-indicator-only quality strategies increasingly costly.
App store discoverability has become harder and more expensive. The cost of acquiring a new user — through paid channels, organic search, or word of mouth — has risen significantly in most categories. At the same time, the window for first impressions has compressed. Users who have a poor first experience are less likely to return, and more likely to leave a negative review that affects future acquisition.
The result is that the cost of a quality failure that reaches users has increased, while the cost of finding that failure before it reaches users has stayed relatively stable. The economic case for leading indicators — for investing in quality signals that arrive before users are affected — has strengthened considerably.
What Leading Quality Signals Actually Look Like
Building leading indicators of quality into a development process doesn't require a fundamental restructuring of how a team works. It requires asking a different question at a specific point in the development cycle: not "does this work?" but "what do real users actually experience when they use this?"
The most useful leading quality signals share a few characteristics:
- They involve real users who have no prior knowledge of the product and no investment in its success. The proximity problem — where internal teams and engaged beta users cannot experience the product as a stranger would — is the root cause of most quality failures that reach the support queue.
- They are structured enough to produce comparable signal across different users and sessions, so patterns are visible. A single user session is anecdote. Structured evaluation across a panel of users is evidence.
- They capture qualitative experience, not just behavioural data. Knowing that a user abandoned a flow tells you something. Knowing what they were thinking when they abandoned it tells you something actionable.
- They happen before launch, or before a significant release, when the cost of addressing findings is lowest. The same issue that takes an hour to fix before launch may take days to investigate, communicate, fix, test, and deploy after it appears in the support queue.
The Cost Comparison That Most Teams Don't Make
Teams that do not invest in pre-launch evaluation often frame the decision as a cost question: evaluation costs time and money that could be spent on development. This framing is incomplete because it ignores the cost of the alternative.
The cost of finding a usability failure before launch is roughly the cost of the evaluation itself, plus the engineering time to address the finding.
The cost of the same failure reaching users includes: the engineering time to investigate and fix, the support load generated while the fix is in progress, the negative reviews that will accumulate and persist, the users who churned before any fix was possible, and the acquisition cost of replacing those users. In most product categories, the post-launch cost significantly exceeds the pre-launch evaluation cost for the same issue.
Integrating Leading Signals Without Slowing Down
The objection to pre-launch evaluation that comes up most often is timing: teams feel they do not have the space in their release cycle to run structured evaluation without delaying the release. This concern is real, but it reflects a specific way of thinking about evaluation as a sequential step rather than an ongoing input.
The most effective approaches to leading quality signals treat evaluation as something that happens continuously and proportionally, rather than as a gate that must be passed before shipping. A small evaluation of a new feature before it ships surfaces issues while they are still cheap to address. A focused evaluation of a specific user flow surfaces the friction that will otherwise generate tickets. Neither requires a full product pause.
The goal is not to replace the signals you already have. Crash reporting, analytics, and support queues remain useful. The goal is to add signals that arrive earlier — before users are affected, before tickets accumulate, before reviews are written — so that the quality picture is complete enough to act on while there is still time to act.
Find what your tickets would have told you — before they arrive.
A Pilot Evaluation puts real users through your most important flows before launch. The findings arrive while there is still time to act on them. Start for $300.