Fraud is a hot topic in market research these days. Go to any conference, and it’s all the buzz. Rightfully so, but sometimes it feels that we, as an industry, aren’t really seeing the forest for the trees when it comes to fraud and data quality. In fact, lately, I’ve found myself in three very different conversations that all circled back to fraud in market research. One was with the CEO of a research firm (a friend, not a competitor or client). Another was with a fieldwork CEO who started his career as a door-to-door interviewer. And the third? It came from something as simple as a text from my car dealership.
At first glance, these might seem unrelated. However, each example highlights an important point: fraud has always existed and will continue to exist, but it’s not the whole story when it comes to data quality. Here’s why.
Fraud Isn’t New
The fieldwork CEO told me how, back in his early days of knocking on doors, he’d notice other interviewers finishing their routes much faster than he did. Later, he realized why: some were skipping the interviews altogether, filling out surveys themselves, and pocketing the $5 meant for participants.
That’s true fraud. And it’s nothing new. Bad actors will always exist, no matter the method or medium.
Bias Is Not the Same as Fraud
Now let’s consider my car dealership. They sent me a text that essentially said the following: “Please give us a 10. If you can’t, call us first.”
That’s not fraud—but it is bias. When a survey is framed like that, the data is automatically skewed toward one result. As my research firm CEO friend and I discussed, grocery store receipts tied to employee bonuses fall into the same category. If a cashier keeps the receipt and fills it out themselves, maybe it drifts into fraud, but the bigger issue is bias creeping into the data.
Fraud and bias often get lumped together, but they’re not the same. Both are part of the larger umbrella of data quality, and both require different approaches.
Fraud Versus Data Quality Issues in Market Research
Here’s the distinction I think we need to make:
- True fraud – This includes survey farms, bots, or interviewers pocketing incentives instead of conducting research.
- Bias and data quality challenges – These include racing through surveys, leading questions, unrepresentative samples, or respondents whose life circumstances change mid-survey. And let’s be real, some of this is our fault as an industry, too. We often create surveys that are too long and not optimized for the respondents taking them.
Too often, everything gets labeled “fraud.” But calling it all the same thing only muddies the waters in market research.
Why Representation Will Always Be Imperfect
Even the Census—the gold standard of large-scale research—isn’t perfect. It’s not fully representative, and it contains its own forms of fraud and bias. For instance, an undocumented Hispanic resident might choose not to participate, even though they’re technically supposed to be counted.
Which brings us to the heart of the matter: research is about managing risk, not eliminating it.
So, What Can We Do as an Industry?
As people used to say about Michael Jordan, “You can’t stop him, you can only hope to contain him.” Similarly, we may not be able to “fix” fraud entirely, but we can manage it. That means:
- Fairly incentivizing panelists — If you pay pennies, you’ll get penny-quality data.
- Choosing sources wisely — If something seems too good (or too cheap) to be true, it probably is.
- Looking at patterns in data — Don’t just focus on open-ends or single datapoints. Look at the whole picture.
- Separating fraud from bias — Treat survey farms differently from rushed responses. Don’t throw out the baby with the bathwater. Instead, you can look for patterns of responses, such as: Are all completions coming in overnight? Do they have the same misspelling of a common word in lots of responses? Is everything coming from blocked or proxy IP addresses? In addition, you can use a tool to see if respondents are copying and pasting answers.
The Bottom Line
Fraud is real. Bias is real. But neither is new, and they are certainly not the same. Instead of obsessing over eliminating fraud, which is impossible, our industry should focus on setting expectations, improving incentives, and managing data quality holistically.
Ultimately, market research is about people. And people are complicated, imperfect, and sometimes unpredictable. Which means the data will always be a little messy, too.
The key is not chasing perfection, but learning to work with reality when it comes to data quality.
For more information on data quality at Research Results or how we can help with your next project, contact Ellen Pieper, Chief Client Officer, Ellen_Pieper@researchresults.com, or 919-368-5819 today.