Customer Surveys' Dark Side

On the whole, the J.D. Power customer satisfaction survey has been a good thing for our industry. Since its introduction five years ago, survey results have risen every year in each city measured. Remarkable. Builders are paying attention. Customers are paying attention, as sales associates report prospects arriving at the office, scores in hand. Service technicians tell of having the scores flaunted at them when something goes wrong. Even suppliers and trades are paying attention. Last month I met with a group of 12 key vendors with a large East Coast builder. The owner of the carpet, tile and flooring company made this statement:

"My 2005 goal is to work ONLY for builders finishing in the top 10 of J.D. Power. Now it's 60%. We have learned that builders who score the best on J.D. Power are also the best ones for us to work with."

At least half the heads in the room nodded as he made the statement. These vendors have discovered that the builders who take care of their customers are the same builders who will take care of them. Meanwhile, a bunch of builders in this particular market, many with household names, soon will be dropped by one of their best suppliers, and they don't even know it's coming.

Builders finally care about customer satisfaction. That seems obvious, but hold on, because as "The Music Man's" Professor Harold Hill taught us, "There's trouble right here in River City." That's with a capital "T," and that rhymes with "C," and that stands for "cheating." Cheating? Do I know that for sure? Can I prove it? Well, no, but as the old adage goes, "Where there's smoke there's fire," and it's getting a bit hazy out here.

Can you really cheat at J.D. Power or any other third-party survey? You can sure try, and depending on how cooperative your customers are, you just might be able to pull it off. The easiest way, of course, is to mess with the database, culling out the "problem customers," preventing them from ever receiving a survey at all. J.D. Power eliminates this by generating its survey lists from public records. So presuming you don't have the skills to hack into the J.D. Power computers and change the data directly, the only other way to artificially change the outcome is to influence, coerce or bribe your customer into letting you "help" with their survey.

The subtle approach is to have everyone on your team repeat this mantra to each customer. "You'll be getting the J.D. Power survey soon, and if there is any reason you can't give us all fives, we want to know right away so we can fix it." This is not cheating per se, and when done in a sincere and professional manner, can even be a legitimate strategy. Who wouldn't want their builder busting its can to get all A's on the report card? But I had a homebuyer tell me recently that after the 25th time she had heard the "All fives mantra," including one service rep who said it at the beginning and end of every conversation, she felt like the builder should change its name to "Stepford Builders."

If you take this road, you walk a very fine line. On one side, you're setting a high expectation for customers to communicate their needs and then wowing them with your prompt and competent response. Walk the other side and you sound phony at best, and at worst, you're unable deliver a straight A performance. That's over-promise and under-deliver, also known as "Customer Service Hell."

Other tales from the front have no such fine line. These stories have customers dropping their survey off at the model or sitting down with service to review it as it is completed. The reported encouragements range from gift certificates to microwave ovens.

There are other reports of builders agreeing to do business with J.D. Power in exchange for high scores. I can say with 100% confidence that this is pure bunk - sour grapes by builders not scoring well. J.D. Power isn't going to put its entire reputation and future on the line to play that game. Get caught once, and it's over.

But manipulation or outright cheating on the local level is not just possible, it is predictable. How can I say that? Would employees actually cheat to get good scores on J.D. Power? Would they go that far? Aren't people more honest than that? Well, that depends. For inspiration, I offer this well-circulated e-mail sent by the now infamous University of Missouri sophomore and blood drive coordinator, Ms. Christie Key, to her 170 sorority sisters at Gamma Phi Beta.

"I don't care if you got a tattoo last week. LIE. I don't care if you have a cold. Suck it up. We all do. LIE. Recent piercings? LIE. We're not messing around! Punishment for not giving blood is going to be quite severe."

Whoa, Christy! The quality of the nation's blood supply has very high external stakes - life or death. At the same time, the internal stakes are quite low - winning the campus blood drive trophy. In other words, there are huge negative consequences for manipulating the results and small rewards for winning. Yet, this misguided young coed cajoled and threatened 170 of her best friends to LIE no less than three times to make the numbers. Imagine what she might do if it really mattered.

Compare that to our survey situation. The external stakes are comparatively low. No one's going to be hurt or killed if the survey results and ultimate J.D. Power rankings are thrown off by local manipulation. Yet the internal rewards for winning can be huge. Monthly and quarterly dinners and tickets are one thing. Now add huge annual bonuses and trips to far away places for the big winners, and people can get very creative but not in the way you hoped. If people could really treat this stuff like just a little icing on the compensation cake, no problem. But I frequently observe people resorting to negative, even destructive behavior to be No.1 and cop the big prize. If not very carefully managed, incentives tied to survey results can do more harm than good, resulting in net loss for the builder.

Therein lays the real problem. If your field people, however subtly, are artificially manipulating the data, you have destroyed the original purposes of the survey, which are:

1) To demonstrate to customers that you seek and value their input;
2) To demonstrate to employees that customer feedback is an operational priority;
3) To confirm by an external source how you are actually doing;
4) To provide data and a baseline for the process of continual improvement;
5) To manipulate your employees into doing the things you could not figure out how to manage them to do.

Did I catch you on number 5? That was a joke, but maybe not a good one because here's a secret. There are plenty of companies who had their employees totally focused on providing total customer satisfaction before anyone ever thought about having a survey on it tied to financial rewards. You have to stop and ask yourself, "Gee, how did they do that?" If you start by looking under the culture rock, you're getting close, and if you'd guess that achieving that focus without tying big incentives to your survey results avoids a lot of problems and bad behavior - you'd be right.

We know that some builders absolutely care about customer satisfaction and some even have a genuine focus on continual improvement. But obsessing on the numerical results is not the same thing, and that's where most builders are today. I'm not saying you can't tie some incentives to your survey, but people ask these surveys to do work they were not intended to do and in the process obscure the very purpose for which they were designed.

For all the benefits our customer satisfaction surveys have wrought, there clearly is a dark side, which you should take proactive steps to avoid. Start by asking one of Dr. Deming's favorite questions, "What is the aim of the system?" Maybe its time you asked that question about your survey. By shedding a little light on the process, you just might keep your people, and your company, out of the dark.