“There is a sample quality crisis in the industry. We’ve had it for years, but it could be ignored because there’s plentiful supply and cheap prices. But that has changed. Quality issues have become so extensive that it’s not unusual to throw away…30% of your sample on average, with 50-60% happening occasionally.” —Lenny Murphy
While sample quality issues have been a longstanding challenge, they have become more pressing due to current supply constraints. This issue has been exacerbated by an overemphasis on speed and cost efficiencies at the expense of data integrity. In a live poll, the majority of respondents said they experienced the following sample-related problems more than once in the past 6 months:
“It’s not just about the explosion of tools… Now we have this whole other component that is AI-driven, making the options even more complex. But there are still questions that are fundamental, like if you’re buying a car—you need to look at quality ratings, reliability, cost, upkeep, etc.” —Lenny Murphy
To combat sample issues, research leaders are using different strategies across the industry:
Alternative Sample Sources: Moving away from traditional panels, researchers are exploring new sample sources to improve quality. This includes partnering with companies that custom recruit respondents and speciality panels that are able to verify their professionals.
Increase Pressure on Sample Providers: Research leaders are starting to hold sample providers accountable for the quality of their deliverables. Leaders can ask providers questions like: Where do you source experts from? Are experts able to self-declare information? How do you verify their expertise?
Building Proprietary Panels: Some organizations are developing their own panels, which allows for greater control over the sample quality and ensures the data meets specific standards .
Automation and Fraud Detection Tools: Implementing advanced tools for automation and continuous quality assessment helps in detecting and eliminating poor-quality data early in the process. The use of sophisticated fraud detection services has seen a significant increase, providing an additional layer of security against data contamination.
Don’t get stuck with research that’s not delivering results. Recognize the triggers for change and foster direct relationships with partners to drive better research outcomes for your organization.
Leading B2B research publication GreenBook partners with NewtonX in latest 2021 GRIT Report. Learn more about our partnership.
read moreWith total confidence in their data, Greenbook rewrote the insights industry narrative.
read morePoor quality data costs the US $3 trillion each year. There has to be a better way, and we believe we've found it. Read on.
read moreJoin Karine Pepin and Jimmy Coonan for a discussion around the pitfalls of using legacy panels and using that knowledge to improve your recruiting and survey results.
read moreListen as Leon Mishkis and Jason Talwar discuss the differences between synthetic and augmented data and a framework for boosting synthetic samples.
read more