We Told This Insurer Their NPS Was Wrong. Then We Proved It.
Telephonic surveys reached 1% of customers and returned an NPS in the 80s. The real number was in the 60s. Here is how the insurer found out.
At a glance
- Telephonic surveys were reaching under 1 percent of customers who renewed, and the client's own indicative figure put NPS in the 80s and 90s.
- That score was not a measurement. It was an artifact of who agrees to a phone survey.
- Moving to digital surveys at the point of interaction lifted the response rate to roughly 8 percent, and daily volume to more than 1,200 responses a day.
- The measured NPS then came in at 63. That drop was the deliverable.
The situation
The insurer had an NPS programme, a healthy-looking score, and no idea it was in trouble.
Feedback was collected by phone. Agents called a sample of customers after key interactions, worked through a script, and recorded the answer. Responses were capped at 200 per touchpoint, so the programme was surveying under 1 percent of the customers who renewed. The client's indicative telephonic NPS sat in the 80s and 90s, which by any published benchmark would place the company among the best-performing insurers in the world.
Nobody inside the business believed it. The scores did not match complaint volumes, lapse rates or what the service teams heard every day. But there was no evidence to argue with, only unease.
Surveying under 1 percent of your customers does not produce a measurement. It produces a portrait of the few people who will stay on the phone with a stranger, which is not your customer base.
What was actually wrong
Two problems, and the second is the one most programmes never catch.
The first is coverage. With a 200-response cap, the programme was reaching under 1 percent of the customers who renewed, describing a rounding error and reporting it to the board as customer sentiment.
The second is bias, and it is not random. People who agree to complete an unsolicited phone survey are not a neutral sample of customers. They skew toward the available, the agreeable and the already-satisfied. Add an agent audibly waiting on the line for an answer, and the incentive to give a generous number rather than an honest one is considerable.
Both effects push the same direction. The score does not drift. It inflates.
What we changed
We moved feedback collection from a sampled phone call to a digital survey issued at the point of interaction, across every major touchpoint rather than a selected few.
Surveys were shortened to what could be answered in the moment. A single scored question with a clear scale, one open comment box, and targeted follow-ups only where the answer warranted them. Delivery was matched to the channel the interaction happened in, so a customer who transacted online was asked online rather than called three days later.
The result
Coverage moved from surveying under 1 percent of customers to surveying at every major touchpoint, and the response rate settled at roughly 8 percent on direct digital interactions. Daily volume rose to more than 1,200 responses.
And the NPS came down. The client's indicative telephonic figure had been in the 80s and 90s. Measured properly on Numr's platform, it was 63.
That is the finding. An NPS of 63 built on 1,200 daily responses across the full customer base is a number the business can act on, set targets against and defend to a regulator. A score in the 90s built on under 1 percent of customers is a number that quietly misleads every decision made downstream of it.
Amitayu Basu, CEO and Co-founder, NumrThe most valuable thing we did for this client was make their score go down. A number that flatters you is worse than no number, because you act on it.
What the drop actually bought
Three things a flattering score had been preventing.
Targets that mean something. You cannot set a credible improvement goal against an inflated score that is not real. From a genuine 63, movement is measurable and attributable.
Segment visibility. At 200 responses per touchpoint there is no statistical room to compare regions, products or channels. At 1,200 a day there is, and the differences between them turned out to be where the actual problems lived.
Trust in the programme. The service teams had known the number was wrong. Correcting it made the rest of the programme credible internally, which is usually the harder battle.
The general lesson
Response rate is not an operational metric to be optimised in the background. A score built on under 1 percent of customers should not be reported as a measurement at all.
If your NPS looks unusually strong and your coverage is low, those two facts are related.
