The number of responses needed for statistically valid customer survey results

It is critical to know what our customers think of living in the homes we build for them and what it was like to be involved in the design and construction process with us. In the case of survey results, this is a lagging indicator but still a track record of how we have done and it is powerful data. The more accurate the data and the faster we gather and act upon this data the more quickly we can listen, evaluate, correct anything that went wrong, prevent it happening again and seek further opportunities to improve.

There are a number of factors to address to ensure that the surveys have an impact. Just one for example is ensuring that we actually ask the right questions in the first place, more is not better! Another is ensuring that your survey results are statistically valid, that is, the responses (sample) are of a statistically representative size.

While getting the most surveys back is the goal, in some cases you don’t need 100% or even 50% to be valid! As the size of the group you want to survey increases, the sample size (the number of surveys returned from the group) falls. You can view this as diminishing returns. At a point you will get the same result if you have 25% response rate as you would with 90%. While we want to get as many surveys returned as possible, the statistics show us that at a certain point we can stop using our resources in getting more completed surveys back and instead focus our time on something else instead! Indeed, we can cause more problems by ‘pushing’ for returns, research shows that satisfaction scores go down the more the respondent feels ‘coerced’ into completing a survey. This can simply be from getting too many reminder emails and certainly from ‘personal reminders’. So some out there may be able to use the time and resources they are currently using to hunt down survey responses to study the data more by using correlation or regression, implement more improvements or spend more time with that customer going over their home plan.

In the columns below, population is for example the number of homes sold in the past year, the sample is the number of households you need to have respond (complete and return) your customer surveys to be statistically valid.

This is at a 95% confidence level (how sure you can be with the information provided) at a +/- 5% degree of accuracy (margin of error- confidence interval). Or that your results will be within 5% of the correct answer 95% of the time you run your survey.

Population Sample (Responses)

10 9

50 44

100 79

150 108

200 132

250 151

300 169

400 196

500 217

600 234

700 248

800 259

900 269

1000 277

1500 305

2000 322

3000 340

3500 346

4000 350

4500 354

5000 356

10,000 369

20,000 376

30,000 379

40,000 380

50,000 381