Wondering Which to Use? A Comparison of Quantitative vs. Qualitative UX Research Methods

In collaboration with Mina Rohani.

User experience research aims to be able to provide information that seeks to provide insight to the user, provide context for usability, and asses potential problems while creating solutions. In short, the aim of user experience research is to gather information. This information can be used to identify facts or patterns, highlight problem areas, and reach conclusions about users and the usability of a product. Research teams then are tasked with deciding which types of research methods, tools, and techniques they are going to utilize to try to obtain their information. Typically, research methods are split into two categories: qualitative and quantitative. In this article, we elaborate on the differences between the two methods, their individual uses, and benefits in user experience research.

Let’s begin by first distinguishing between the two. One of the main differences that exist between quantitative and qualitative UX research is the type of questions that each seeks to answer or the type of information they hope to collect. The simplest way to remember the difference between the two thankfully lies in their respective names; quantitative research is focused on the quantity of data, whereas qualitative research is aimed at the quality of the data. Quantitative data provides information about quantities that can be measured numerically; qualitative data is about qualities of data, or the ability to provide robust, in-depth analysis that cannot be measured numerically. Essentially, quantitative research seeks to answer research questions such as “how many?” and “how much?”, where qualitative data seeks to answer the research question of “why?”

Quantitative research uncovers patterns in research and uses measurable data to formulate facts whereas qualitative research is primarily exploratory research and is used to gain an understanding of underlying reasons, opinions, and motivations. Hence, quantitative methods are usually much more structured compared to unstructured or semi-structured qualitative methods.

How to Choose?

The world of user experience research benefits from the usage of both research methods, but sometimes it is tricky to try to figure out when to use one research method over the other. One of the simplest ways to decide is to determine what are the goals of the research project. If the desired outcome of the research project is to listen and to learn from the user, then choosing qualitative research methods might be best. If the goal of the research project is to measure the usability of a product or a design, then it might be better to consider using quantitative methods.

When to Use Quantitative UXR?

The type of quantitative research methods that are used within the world of UX can vary from analyzing data, trends from web analytics to a close-ended survey, user product testing, and benchmark testing. Quantitative research is helpful because it is able to provide information that can be used to assess the overall usability of a system. Typically when using quantitative methods, there will be a larger sample population compared to qualitative methods. Since quantitative data is focused on generating numerical results, the results can ultimately prove the statistical significance of the results.

Another criteria to help us decide which research method to use is related to numbers versus stories. If your goal is to compare products or designs or to compute expected cost savings from design changes, you can use a quantitative research method (e.g., A/B testing, card sorting, tree testing, eye tracking, heat maps, usability tests, surveys multivariate testing).

The next criteria to consider when selecting a research method to use is whether it is tangible or not. Sometimes it is not easy to recruit a large number of participants in order to conduct a quantitative project, and sometimes a timeline doesn’t allow for in-depth qualitative research. We also qualitatively study people when we have no design at all or we are interested to learn how people work and live in order to better understand the environment in which a user exists in.

When to Use Qualitative UXR?

Qualitative UX research methods are best used when the research team or client is focused on gaining a deeper and more insightful understanding of the typical user’s daily experiences, motivations, and struggles. Interviews, focus groups, and ethnographic field studies are just some of the ways in which a UX research team can utilize qualitative research methods. As mentioned earlier, qualitative research seeks to answer the “why” questions posed by the research team and focuses on getting more in-depth responses from fewer participants. Qualitative research can be used to help inform design decisions, identify potential usability issues, as well as provide an overall positive or negative assessment on user experience.

There are a few criteria that can help us decide which UX research method to use. First, research can be either behavioral or attitudinal. Behavioral research observed by researchers whereas attitudinal research is self-reported by the user. When your research question has to do with what people actually do or whether something is discoverable, findable, understandable, or usable, it's most informative to quantitatively observe it. If you're wondering what people have to say (e.g., what they believe, like or dislike something) that can be researched with self-reported or qualitative methods such as interviews and focus groups.

Conclusion

While each research method has it’s own strengths, it is important to take the time to consider which method to use in order to get the best possible data for a given research goal. Additionally, quantitative and qualitative research methods work together in order to fill in the gaps of what the other one missed. Both quantitative UXR and qualitative UXR are focused on achieving the same goal: uncovering the truth and providing supporting evidence.