How to Run a Voice of the Customer Program

A voice of the customer program is the arrangement by which a company collects what customers say and turns it into decisions. Most companies already collect what customers say. The work goes wrong after the response arrives, when someone has to decide whether the number means anything, which issue to work on, who calls the customer back, and whether the fix paid for itself.

This page is built around the decisions a leader has to make when setting up or repairing a program, and each section links to the page that carries the detail. If you are choosing a platform while you build the program, the CX platform scorecard turns the same decisions into questions to ask a vendor.

The short version

• Name the decision before the first survey goes out. A program that does not feed a named meeting produces reports nobody acts on.
• Ask about one event soon after it happens, and about the relationship on a schedule. They measure different things and will not match, and the gap between them is not a target.
• 385 responses buys a margin of error of about plus or minus 5 points on a single proportion. Comparisons and changes need more.
• There is no good response rate. Above 50% and around 3% are both normal, depending on the touchpoint and on whether something just happened.
• Look at all the issues together, not one at a time. One at a time makes everything look bad and can blame the wrong thing.
• Comment topics add to more than 100%, because a comment can mention several things.
• Answer the individual customer in a day, and fix the process in a quarter. The two loops have different owners.
• Count what customers did next. It is the strongest evidence a CX program can offer. It does not establish cause, so the limit travels with the number.

What does a voice of the customer program have to produce?

The program has to produce a named decision, made on a schedule, by a named person. Listening is an input to that decision and does not count as the output.

Before the first survey goes out, write down the decision the results will feed. The quarterly staffing review might use the transactional score by branch. The product roadmap meeting might use the drivers from the relationship survey. Name the person who owns each decision and the date it gets made, so there is someone to ask when nothing changes.

If you cannot name the meeting where a sharp drop in a score would be discussed, and the action that would follow it, the program is not built yet.

One thing to remember. Write the decision down before the questionnaire. Everything after this, from the scale to the alert rules, is easier to settle once the decision is named.

What should you ask, and when?

Ask two different questions at two different times, because they measure two different things. A relationship question asks how the customer feels about the company as a whole. It goes to a portion of customers monthly, quarterly or annually, or on the anniversary of their purchase. A transactional question asks about one event, such as a delivery or a support call, shortly after it happened.

The relationship score moves slowly. The touchpoints move it, each contributing a different amount on its own schedule, along with things sitting outside the program entirely, such as brand. The transactional score moves fast, and it belongs to the team that owns the touchpoint.

The two will not match, and they should not. A customer can rate a support call highly and still plan to leave, so a gap between the two is expected. It has no standard size and no fixed direction, and it should not be set as a target. The page on transactional NPS versus relationship NPS explains the difference, and the page on what NPS is and how to calculate it covers the arithmetic.

The survey type decides the question. The 0 to 10 recommendation question belongs in the relationship survey and goes first there, and it does not belong on a transactional survey. An effort question is the one for a transactional survey, where the customer has just done something. A satisfaction rating suits a short in-app pulse. Pick one per touchpoint and keep it stable, so periods can be compared.

One thing to remember. Start with the transactional or conversational survey, because it is the one a team can act on. Add the relationship survey once that is running. Add a competitive benchmark only if you need it.

How should you design the questions?

There are four CX surveys, and each has its own design. A conversational survey is one rating question followed by a conversation the model runs, and it is the one to use when you need the reasons behind a score and have no ready-made questionnaire. A transactional survey follows an event and asks about that event. A relationship survey goes out on a schedule and asks about the whole relationship. A panel benchmark asks a recruited sample about your company and your competitors.

A 5-point scale fits a phone screen without shrinking. A 7-point scale spreads answers out more. A study by Dawes, published in the International Journal of Market Research in 2008, put the same eight questions to matched samples at five, seven and ten points, and found that the five-point and seven-point versions produced identical means once rescaled. He tested means only, and the top-box and net scores most CX programs report were not tested. Use one scale length across a questionnaire, and change it as rarely as you can, because a change breaks the trend line your teams report. The comparison of 5-point and 7-point Likert scales sets out the trade-offs, and the page on what a Likert scale is covers the format itself.

The rating question goes before the open comment box, so the comment explains the rating instead of shaping it. Account details should come from the customer record, so nobody is asked what you already know.

A customer who receives a survey after every small interaction stops answering, and the people who keep answering are no longer typical of the base. Set a rule for how often one customer can be asked and hold every team to it. The detail is on the CX survey design page.

One thing to remember. Decide the scale before launch and hold it across the whole questionnaire. A change mid-program breaks the trend your teams report, and the old and new periods will be argued over for a year.

How many responses do you need before the result means anything?

About 385 responses give a margin of error of roughly plus or minus 5 points at 95% confidence. That figure applies to a single proportion, such as the share of customers who chose 9 or 10, drawn from a very large population.

The figure moves in both directions depending on what you are measuring. A finite customer list needs fewer responses. A net score like NPS needs more, because it is the difference between two proportions and each carries its own error. A change between two periods needs more again, since both readings carry error. A comparison between segments, such as two regions, needs enough responses in each, and the smaller segment sets the limit.

At 100 responses, an NPS reading carries roughly plus or minus 15 points on a typical 50 / 30 / 20 split, and about 19.6 points on a fully polarized one. Only a very large move will register at that size. A team that reports a small change on 100 responses is reporting something the sample cannot tell apart from noise.

No sample size repairs a self-selected sample. If the people who answer are mostly the very happy and the very angry, collecting more of them makes a biased result more precise without making it more truthful. The page on how many survey responses you need has the working for each case.

One thing to remember. Size the smallest group you plan to report on, not the total. A regional cut of 80 responses cannot carry a regional decision, however large the overall sample is.

What response rate should you expect?

Response rates in Numr client programs have run from above 50% to around 3%, and both ends were healthy. In those programs, a survey sent shortly after a premium vehicle purchase returned above 50%, and a relationship survey with no triggering event returned around 3%. Each of those is one delivered program rather than an average across a category.

Response depends on how much the customer cares about the category and on whether something just happened to them. A person who has just spent a large sum on a car has a lot to say and a fresh memory. A person who receives an annual survey from a utility has neither. Treating 3% as a failure to be fixed with reminders would be a mistake.

A response rate is also the end of a funnel. Records are excluded before anything is sent, some messages never leave, and some that leave are never delivered. The same program can report one defensible rate against every record it started with, and a higher one against the invitations actually delivered. Neither is dishonest, and they are not comparable, so a rate should always carry its denominator. Count delivered rather than sent, or a delivery problem will look like a customer who did not care. The page on what a good survey response rate is covers what to expect, and the survey response rate page covers how to calculate and report it.

One thing to remember. Agree the denominator once, in writing, before anyone reports a rate. Two teams dividing by different stages will report different numbers for the same program.

How do you tell which issues matter?

Run one driver analysis across all the issues at once. Driver analysis is a statistical method that estimates how strongly each issue is linked to the overall score once the other issues are accounted for.

Looking at one issue at a time misleads because customers raise several issues at once. The customer who complained about delivery also complained about billing. If each issue is examined alone, that one customer is counted twice, and both issues look bad. When two issues almost always appear together, such as long wait times and unhelpful staff, a simple comparison cannot tell which of them is linked to the score. It may blame the wrong one.

A driver chart tests all the issues at the same time. It can also hold an issue back, and it should say why. The method holds an issue back for one of four reasons. Too few customers raised it. There was no usable comparison group, because nearly everyone raised it and almost nobody is left to compare against. It failed the evidence test, meaning the link to the score could have arisen by chance. Or it overlapped so heavily with another issue that neither could be separated from the other. A held-back issue should be listed with its reason, so the team knows it was considered. Where an issue is held back for overlap, read the one that remains as carrying some of the other's weight.

A driver chart shows a relationship between an issue and the score in the data you have. It never forecasts what fixing that issue will do to the score. Numr's account of testing its own driver analysis covers what the method can and cannot do, and the page on reading a CX dashboard shows how the chart looks in practice.

One thing to remember. Customers usually give several reasons for being unhappy at once. On a checked list of reasons, the driver chart works out how much lower the scores are among the customers who raised one particular reason, compared with customers who did not raise it, taking their other reasons into account.

What should you do with the comments?

Text analytics places open-ended responses into categories the company defines in advance, such as billing, wait time, or staff attitude. Before trusting the sorting, read a few hundred comments yourself. Each comment is counted in every topic it mentions. A comment that says the agent was polite but the refund took weeks counts once under staff and once under refunds. Topic shares can therefore add to more than 100%, and they should not be read as slices of one pie.

Your company defines the categories. Start from the issues your teams already recognize and keep a catch-all for everything that fits nowhere. When the same new problem keeps landing in the catch-all, give it a category of its own.

Each category is reported as a share of the people who gave a comment, not of everyone who answered the survey. Those are different numbers, and a room full of executives will assume the larger base unless somebody says otherwise. The text analytics for open-ended feedback page explains how the sorting works and how to read the output.

One thing to remember. Say the base out loud whenever you show topic shares. They are shares of the people who commented, and they add to more than 100%.

What happens after the feedback arrives?

Two loops close, at two different speeds. The inner loop fixes one customer. A low rating or a flagged comment creates a ticket, someone contacts the customer, and the issue is resolved or explained within hours or days. The outer loop fixes the business. Patterns across many tickets feed a change to a process or a product, and that takes weeks or months.

The inner loop needs rules written down before the program starts. An alert rule sets what triggers a ticket, such as a rating below a set point or a word in a comment. The customer should have agreed to be contacted before anyone calls. Numr's general recommendation is to answer within 24 hours, and faster for a critical customer or for someone saying a closed ticket was never really fixed. Statuses have one job, which is to separate what is waiting on you from what is waiting on the customer. With a single open state, nobody can see which tickets are stuck on your side, and the queue grows. An escalation path names who takes over when the frontline owner cannot resolve it, and every alert has an owner, either a named person or one derived from a column in the response data.

The outer loop stalls when the fix belongs to a team that never saw the survey. A standing review that takes the drivers and the ticket patterns to the owners of those processes, with a written record of what was decided, keeps it moving. The close looping page describes both loops and how to set them up.

One thing to remember. Give every alert an owner before launch, either a named person or one derived from a column in the response data, and make your statuses separate what is waiting on you from what is waiting on the customer.

How do you tell whether it paid for itself?

Nobody can put a price on an experience. What you can do is count what customers did next: a repeat purchase, a renewal, a second product, a referral.

Take everyone who answered the transactional survey in a period. Split them into two groups: the ones who rated it at the favorable end, and the ones who rated it at the unfavorable end. Leave the middle out. Follow both groups for a set time and count how many took the next step. Divide the favorable group's rate by the unfavorable group's rate. That figure is the multiple.

Leaving the middle out widens the gap, so the multiple runs higher than a comparison against everyone else would. Decide where each group starts before you run the numbers, and say where you drew the lines when you report it.

The multiple measures a difference in behavior between two groups of customers. It does not establish that the experience caused the renewal, because the customers who rated it well may differ in other ways too. It is not an ROI figure in the accounting sense either, since it does not net out the cost of the program or the fix. Stated with those two limits, it is a number a leader can take into a budget meeting. The post on counting CX ROI walks through the calculation.

One thing to remember. Decide where the favorable and unfavorable groups start before you run the numbers. Move those lines and the multiple moves with them.

Where do these programs fail?

Five failures account for most stalled programs.

• Surveys nobody acts on. The program collects and reports, and no decision was ever named for it to feed.
• Comparing two scores that were never comparable. A different scale, question, moment, or denominator makes the change meaningless.
• Measuring only at the relationship level. The score moves, no team can see which touchpoint moved it, so nobody acts.
• Treating the score as the goal instead of the decision it feeds. Frontline teams learn to ask for high ratings, and the score rises while the decision gets worse.
• Acting on an analysis nobody questioned. A chart goes straight into a roadmap, and nobody asks what it left off, or why.

One thing to remember. Most of these failures start before the first survey goes out. They come from what the program was built to feed, and not from the questionnaire everyone argues about.

Where does AI belong, and where does it not?

AI can hold the conversation, asking follow-up questions a fixed questionnaire cannot ask, until the real problem is on the table. It can also sort comments, so every response is categorized rather than a sample of them, and a problem nobody expected keeps landing in the catch-all until someone notices it.

AI cannot tell you what your customers think without asking them. A model trained on other people's data does not know your customers' reasons. Behavior data shows where a customer stopped, and the reason still has to be asked for. The conversation still has to happen, and the program is how you hold it at scale. Where AI sorts or summarizes, a person should approve the categories first and check a sample of the output against the raw comments.

One thing to remember. Whatever AI sorts or summarizes for you, have a person approve the categories first and check a sample of the output against the raw comments.

A starting checklist

1. Write down the decision, the meeting it feeds, the person who owns it, and how often it happens.
2. Pick two or three touchpoints that matter to that decision.
3. Choose one rating question per touchpoint, and one scale for the whole program.
4. Check the list you are sending to. Records excluded before send, and messages that never arrive, both surface later as a low response rate with no explanation attached.
5. Set the alert rules and the owner before launch.
6. Agree how response rate will be quoted, with its denominator, so two teams do not report different numbers.
7. Take your first response-rate target from your own first cycle. There is no outside rate to aim at, and the first cycle shows what normal looks like for your program.
8. Book the review meeting now. Without a standing review, results get circulated and nothing is decided.

If part of this list depends on software you do not have yet, the CX platform scorecard lists what to ask a vendor at each stage of maturity, from measuring a score to connecting it to money.

Frequently asked questions

What is a voice of the customer program?

It is the arrangement by which a company collects what customers say and turns it into decisions. The decision side is a named meeting, on a schedule, with a person who owns the outcome.

How many survey responses do I need?

About 385 responses give a margin of error of roughly plus or minus 5 points at 95% confidence for a single proportion from a large population. A net score, a change between periods, or a comparison between segments needs more. At 100 responses an NPS reading carries roughly plus or minus 15 points on a typical 50 / 30 / 20 split.

What is a good survey response rate?

There is no single good rate. In Numr client programs, a survey sent shortly after a premium vehicle purchase returned above 50%, and a relationship survey with no triggering event returned around 3%. Each is one program rather than an average, and both are healthy for what they are. Always quote a rate with its denominator.

Should I use NPS, CSAT, or CES?

The question follows the survey. The recommendation question belongs in the relationship survey, an effort question belongs on a transactional survey, and a satisfaction rating suits a short in-app pulse. Pick one per touchpoint and keep it stable.

Why are my relationship NPS and transactional NPS different?

They measure different things. The relationship score reflects the whole relationship, built up from every touchpoint plus things outside the program such as brand. The transactional score reflects one event. A gap between them is expected, has no standard size or direction, and should not be set as a target.

What is driver analysis in customer experience?

It is a statistical method that estimates how strongly each issue customers raise is linked to the overall score, once the other issues are accounted for. It shows a relationship in the data and never forecasts the effect of a fix.

What does closing the loop mean in customer feedback?

It means acting on feedback at two levels. The inner loop contacts the individual customer, ideally within 24 hours, and resolves or explains the issue. The outer loop takes the patterns across many tickets and changes the process that caused them.

Numr designs and runs programs of this kind, and the pages linked above are the working detail.