3 Tips to Writing Better Survey Responses

In collaboration with Annabell Ho.

Surveys are often regarded as being easy ways of collecting large amounts of data. You put together your questions, design your survey, and boom- it can be distributed and accessed online by hundreds upon thousands of participants. While it seems easy enough, there is actually a great deal of design thinking that goes into the creation of a well-designed and efficient survey. In order to make sure that once your survey goes live you generate the best possible results, there requires a hefty amount of thought, design, and care that goes into the construction of a survey. A bad survey design can lead to bad data by causing participants to be unsure of how to answer questions, or not providing enough options to accurately capture the participants' true experience.

In terms of survey design, there’s a lot of advice out there on survey design that focuses on the survey questions themselves (for example, to stay away from leading questions). Not as much attention has been paid, however, to the response options that are listed with each question (for example, “not at all” or “dislike a lot”). Therefore, designing better survey response options can help ensure that your survey is able to generate the most accurate, useful, and richest data. Here are three tips on how to create better survey response options and yield the best possible data.

Using Equally Spaced Options
A commonly used type of response option scale is the Likert scale, which encompasses a set of response options that cover a range of one dimension (“not at all effective” to “extremely effective”) or two dimensions (“dislike a lot” to “like a lot”). When using scale questions, it is critical to ensure that the response options throughout the survey are equally spaced from each other. This can be tricky if you are using words as opposed to numbers as the form of measurement on the scale. Having everything be equally spaced from each other on the scale allows the participant to feel confident in selecting an answer they feel best encapsulates their feelings or experiences. If spacing is uneven, your scale may not represent respondents’ true response very well and lead you to assume that respondents are less or more extreme than they actually are.
Avoid Using Multiple “Not Applicable” Options by Using Skip Logic
Sometimes you might have a set of questions that only apply to a subset of users (for example, only users that have used your product before). But showing all questions to all users with a “not applicable” response option for each one lengthens the survey and causes respondent fatigue, and also allows for satisficing, which is when respondents skim through surveys and pick the easiest response option they can find. Instead, you can use skip logic. Skip logic is a design feature that allows for the creation of a custom path through a given survey that is based on participant’s answers to certain questions. Skip logic is also often known as “conditional branching” or “branch logic”.
Skip logic is a very important feature to add to your survey responses because it can then change the question or page that a participant sees based on their answers. For example, if a participant responded “not applicable” to a certain question, skip logic can then eliminate the rest of the questions surrounding that topic, so then the participant can focus on answering the questions that do apply to them. Implementing skip logic helps to tailor the survey questions to the particular participant, and ultimately lets you avoid causing respondent fatigue or satisficing by presenting “not applicable” options multiple times.
Avoid Using “Yes or No”
One key piece of creating better survey responses is avoiding using “yes” or “no” options for a question. “Yes/no” responses provide the opportunity for participants to satisfice by quickly choosing all the “yes” options, inadvertently introducing bias into your survey results. Additionally, research has shown that participants are more likely to feel the need to agree with a survey question and select the “yes” answer- even if this is not accurate to their true sentiment. Rather than using “yes/no” options, try to write out both the affirmative and disconfirmation responses without using the words “yes” and “no”. For example, if your question is, “have you gone to the gym today?”, response options can be “I have gone to the gym today” and “I have not gone to the gym today” rather than “yes” and “no”.