Rise of the hobbyist respondent

Original blog by former Foolproofer Rachel Orton.

The research world has seen increased numbers of people making research participation a paid hobby. While the majority of reputable online panels have processes in place to prevent respondents from being over used, they cannot control the number of surveys provided by other companies that the respondent may choose to participate in.

This rise in hobbyist respondents has been fuelled by the advent of sites containing links to companies that pay for participation in online surveys. This realm is becoming so sophisticated there are even review sites for paid surveys, where professional participants rate and review online surveys on topics such as the quality of incentives and overall experience.

Some would argue that this is a good thing. Increasing numbers of people willing to participate in market research is a researcher’s dream. Witness interviewer’s frustrations as people shuffle past, eyes averted, or mumble ‘I haven’t got time’ before ambling into the nearest Starbucks. Traditionally, focus groups have always offered incentives to encourage participation. So who is it really hurting?

The industry and our clients, chiefly. Obvious serial respondents have traditionally been excluded from focus groups for a reason. Familiarity with research, its language and processes skew the results and can bias other group members. The result is not true, honest opinion; it is what the respondent thinks the moderator wants to hear. Equally, if they have a glut of online surveys to complete it is unlikely they will take their time and give measured, thoughtful answers.

So, how is it possible to prevent respondents from signing up to multiple panels? It isn’t, is the answer. But a glimmer of hope lies in the formation of the US-based Online Research Quality Council, established by the Advertising Research Foundation.

The aim of the Council is to draw up a set of standards for buyers of online research, to ensure quality control across the board of companies offering online surveys. While this may not curtail multiple panel membership, by encouraging companies and their panel managers to scrutinise their own panels more carefully, and focus on the survey-taking behaviour of respondents, they can help ensure there are no hobbyists lurking.

As for clients commissioning research, the safest way to ensure the results are as realistic as possible is to use online surveys run by professional market research companies, with screening processes in place to minimise this problem.

Responsibility for tackling the problem of multiple panel membership lies with the providers of online surveys in employing safeguards to weed out hobbyists, and by joining forces to employ industry wide standards of quality and professionalism. This seems to be the only way to help eradicate the problem before it spreads further, and to safeguard the increasing popularity of online surveys.

No comments yet.