A key issue growing in the market research industry is sample quality. Both marketers and field service providers are experiencing higher-than-normal rates of data degradation in the forms of bot responses, fake respondents, and fraudulent accounts. There is also the increased use of “professional” surveyors—respondents who simply want to collect as much incentive as possible through falsifying personal information on qualifying questionnaires. Marketers require quality, reliable data insights to develop solutions for their marketing and research initiatives.
As I spend time with clients each week discussing possible methodologies for their research, I am routinely asked the question: “How do you find your respondents?” Most clients will use three words synonymously to describe respondents: Sample, panel, or database. Although they may sound the same, each is quite different in their development, management, and application. Additionally, each has its own specific risks regarding data quality and the mitigation policies required to decrease poor sample quality.
What is a Sample?
A sample can be simply defined as a smaller portion taken from a larger whole for measurement and testing. In the market research industry, this is a list of individuals selected from a larger pool for the purpose of asking questions to extract data insights. Samples can come from government entities, private businesses, or vendors that provide lists of individuals to companies at cost. Examples include consumer lists, voter registrations, and customer/business lists.
Political polling relies heavily on utilizing random samples to determine current key voter trends. Thousands of surveys are conducted each week through emails, phone calls, postcards, and text messages using state voter samples.
Challenges with Samples
Recently, market research companies have relied on sample providers to build up their survey pools for quantitative and qualitative studies. Management of these incoming samples is a huge challenge since each vendor must be assessed and properly screened before incorporating their data to ensure that all data points are accurate, updated, and reliable. Additionally, the data must also go through a vetting process for quality assurance. The biggest challenge with samples is data quality. Generally, samples are managed more as a list of names rather than a database of individuals. When you receive a sample, you can’t help but ask questions like:
- “Is all the contact information attached to the person’s name 100% accurate?”
- The answer is yes—many companies can easily call a phone number to get a live answer or send an email to determine if it bounces.
- “Are the phone number and email address active AND associated with the individual, or will I end up with a wrong number and an unopened email?”
- “How do I know that the survey is being completed by a real person?”
Samples have their place in the research industry but also pose the highest risks for poor data quality. Prior to accepting a sample, ask questions about where it came from, how it is currently being managed, and the level of accuracy that can be guaranteed.
What is a Panel?
Panels are very different than samples. A panel is a group of individuals who have volunteered to participate in market research studies. The objectives, sizes, and lengths of a research panel are infinite in variety. The term “panel” has historically been applied to a specific group of people participating in a specific research study. However, in recent years, “respondent panel” has been used to describe any in-house respondent pool.
A Sample vs. a Panel
The biggest difference between a panel and a sample is the voluntary action of the respondent. All panelists have volunteered their information to be contacted for market research, whereas individuals in a sample tend to be random and unaware of the reason or purpose of the research.
- Panels also consist of individuals who have been screened and qualified to participate in the research study.
- A sample tends to provide individuals without any pre-screening of demographic or personal criteria.
Panel Providers
There are over a dozen companies that exclusively provide marketers access to their panels for market research. These panels have thousands of individuals who have signed up to participate in online surveys and market research. Additionally, panel providers will partner with other companies to continually build up their respondent pool for their clients’ research.
Challenges with Panels
Due to the high number of participants and low cost of research, panel providers have similar challenges in data quality as sample providers. Most of the research process is automated, including:
- Respondent registration
- Screening
- Invitation
- Survey completion
- Compensation
Therefore, very little human touch is used for identity verification and data validation. Generally, panels are managed more as a list of respondents than a database of individuals. Although panel providers can be a strong option in major markets across the globe, they struggle with providing adequate, quality respondents in smaller cities, towns, and rural areas.
Use Cases for Panels
Panels are most useful and beneficial for marketers looking to conduct research in high-frequency settings. They can also be a great option for continual research that builds off previous sessions to gain deeper insights.
However, many marketers have gravitated toward panel providers for quick, turn-around quantitative (and qualitative) research. The results have spurred a new movement within the industry to initiate solutions in our technology and data collection processes to reduce fraudulent activity and poor data quality.
Not everyone conducting research needs to consider utilizing a panel. Marketers should determine if their research process would benefit from a panel and how they want to develop it—either building one in-house or utilizing a field service provider to assist.
What is a Database?
By definition, a database is simply an organized collection of data structured to be easily managed, accessed, and updated. Both panels and samples are, by definition, databases, but how they are managed, accessed, and updated are their distinguishing factors.
What is the difference between a database and a panel in relation to respondent pools?
Both require participants to volunteer to participate. Both pre-screen and qualify participants. Both have information other than basic demographics for clients to cater to their study. Both can be used for qualitative and quantitative studies. However, databases have unique characteristics that differentiate them from panels and samples.
Databases rely on:
- Mutual communication and respect between the respondents and the research vendor.
- Human interactions that go beyond automated text messages and emails.
- Continually updated and validated data with some amount of human touch.
- Confidence and trust that respondents’ personal data is kept safe, secure, and shared with no third parties.
Advantages of Selecting a Market Research Vendor
Databases are managed as a relational database of individuals. The biggest advantage in selecting a market research vendor that utilizes a respondent database is the relational aspect between the respondents and the vendor. When a respondent receives a phone call, text message, email, direct message, or social media message from the vendor, they know who the vendor is because of their previous communications. In turn, this creates trust between the respondent and the vendor, which ultimately results in better quality respondents, greater show/participation rates, and more meaningful insights for clients.
Market research vendors that manage their own respondent databases are a great option for research studies that require excellent data quality and insights. Although more prone to qualitative research, databases are excellent for all types of market research studies.
A common question we get asked from various market research clients: Is managing a database more expensive than a panel or sample? Will it cost more to utilize respondents from a database versus other alternatives?
Yes, it will! But what do you receive in return? Quality, articulate respondents that translate into high quality data! Companies that utilize databases for their field services rarely run into issues of fake accounts, fraudulent activities, professional respondents, or BOT responses. Time, energy, and resources are allocated to properly manage the data integrity of all the respondents in the database.
The Best Option for Your Upcoming Research Study
- Samples are great for studies requiring data from the general population. These types of studies use sample lists for mass emails, text messages, mailers, and phone calls.
- A successful phone vendor will use a CATI (Computer Assisted Telephone Interviewing) system to tackle the high volume of telephone numbers.
- Samples are also beneficial for studies where a client may want the research executed as a double-blinded study—when you want both respondent and vendor to be unaware of who they are contacting for the research.
- Panels are best suited for quick turn-key quantitative surveys or long in-depth research studies that rely on time as a key component to the data collection.
- Vendors that manage research panels can deliver cost-effective survey results within days without dealing with respondent communications or incentives.
- Additionally, they can create short or long-term panels of respondents willing to provide daily or weekly feedback on a product or service over an extended period of time.
- Vendors that manage their own databases are best for any qualitative research study that requires quality, articulate respondents to show up, participate, and stay engaged.
- They are also great for hybrid studies that incorporate aspects of both qualitative and quantitative research, such as an online bulletin board.
Benefit from an Unmatched Database with Eastcoast Research
Eastcoast Research has managed its own relational database of respondents for over 50 years. Our respondents are neighbors and community members in the cities and counties we call home. The relational network we have created over the years makes our respondent database one of the largest and most reliable in all of North Carolina. They know who we are, and we know who they are. The relational aspect is what ensures you receive the best respondent for your study.
Trust Eastcoast Research with your next market research recruiting project. Request a bid today!
