In order to gather reliable and actionable conclusions from your data, you need to first make sure you are gathering quality responses. 

Research can often become muddy with bad responses that leave you with more follow-up questions than answers. Making sure your survey is clear and unbiased is imperative. 

It is also equally important to make sure your respondents are attentive and motivated to provide you with their best, and most truthful responses. 

Below are 5 best practices that will allow you to gather the best quality respondent data:

1) Provide thorough directions and ask straightforward questions

At the beginning of the survey, thank respondents for their time and let them know how long the survey should take so they can remove any distractions and prepare to give the survey their full attention. 

• When providing respondents with the estimated completion time, make sure it is accurate, or even a little exaggerated, to avoid creating unrealistic expectations that can frustrate survey takers, resulting in more survey abandons.

Reassure respondents of the confidentiality of the survey. 

• Explain that their responses and all personal details they share will be kept confidential and be analyzed as a part of a larger group of data. Mention that no sales attempt will result from their participation in the study.

When introducing a new section of a survey write out concise instructions on what you are expecting their responses to revolve around.

• “For this next section we would like you to think about any interactions you may have had with Brand X in the past 6 months. Please carefully read and answer the following questions.”

It is important to make sure the questions you are asking respondents are clear and to the point. Do not leave questions up for interpretation. 

• For example, asking, “How many of these brands have you purchased?” may leave respondents confused about the time frame. Ever purchased? In the past year? Past 6 months? Past week? Leaving out specific details like time can lead to inaccurate results.

When asking open-ended questions make sure the question cannot be answered with a yes or no response. 

• Ask respondents to be as detailed as possible to avoid one-word responses. If possible, at a minimum character count to open-ended questions.

 2) Ask unbiased questions

Make sure the questions you are asking are not biased. 

• Sometimes it is difficult to catch question bias as it typically can cause data to skew the way you’d prefer it to. While this may seem beneficial at first, skewing questions leads to biased insights and a misrepresentation of reality. 

• Conclusions that are drawn from the analysis will not accurately reflect your target audience, leaving you with wasted resources and inaccurate predictions, which could also lead to ethical concerns.

Instead of questions like:

“Do you agree that Brand X’s customer service is excellent?”

Try:

“How would you rate your experience with Brand X’s customer service?”

3) Randomize options and use multiple question types

Randomizing options helps reduce order bias and increases the validity of responses. Respondents are more likely to carefully read options that are randomized rather than listed in a static order or even alphabetical. This will also improve user experience by keeping respondents alert and aware of the questions being asked.

Randomization also reduces the likelihood of response patterns, like the first option being selected the most often. This helps to ensure that respondents are accurately selecting the option(s) that relate with them most. 

Try to incorporate various types of questions such as multiple select questions, open-ended questions, likability scales or grids, and ranking questions. Add visuals where necessary to keep respondents entertained and alert. When possible, this also allows them to better visualize the questions being asked, leading to more accurate and detailed responses.

Asking respondents only single select questions can cause user fatigue and lead to respondents paying less attention to questions and selecting options at random.

Including red-herring options into several questions can help identify respondents who are not paying close enough attention to the survey. Including options that either don’t make logical sense or instruct respondents what answer to choose (e.g. ‘Choose 4 for this question’) can remove these poor quality respondents from finishing the survey, which reduces the time you’ll need to spend cleaning the final data. 

4) Motivate through incentives

Let’s face it, most respondents are not willing to use up their valuable time to take surveys for free. Incentives are a great way to increase the completion rate of your survey and keep respondents honest and attentive. Surveys that are not incentivizing respondents leave the data open for questioning. Did respondents really read the questions? Is this really how they feel? Is this data even reliable?

Rewards or incentives keep respondents honest and willing to take their time to read and respond to questions. If the survey is going to take longer than 15 minutes, try increasing the incentive to motivate respondents to finish the survey.

At the beginning of the survey, make sure to explain that the incentive will only be rewarded to respondents who complete the survey with honest responses. Reiterate that any nonsense responses will be removed and will not qualify for compensation.

5) Testing the survey and quality checks

Arguably the most important part of creating a survey is the review process. 

This refers to testing surveys multiple times through for spelling and logic errors, but also helps improve the clarity of questions. If you are having trouble understanding what the question is asking, odds are respondents will be confused as well.

Quality checking data is important to confirm that fielding is going as planned and there are no logic errors that were missed in testing. 

It is important to check the data as the survey is live. Do not wait until the survey is out of field to check data quality. Checking the data while the survey is live allows you to review responses and make sure questions are clear and being relayed as expected. 

Additionally, if there are respondents who are not paying attention or providing you with irrelevant responses, you can remove them and re-field for more respondents or adjust your target audience.

There’s an art to conducting high quality market research, and utilizing these five techniques can help ensure that your survey is engaging, properly incentivized, and composed of high quality responses. 

If you ever need assistance with your survey development process, remember that Lab42 offers both survey consultation and survey writing as a la carte services. We want you to be confident in the data you’re collecting - because there are few things worse than getting survey data back and realizing it’s riddled with logic inconsistencies and bad quality responses.