When to Use Quantitative Research & Analysis for a Project?

With each User Research project, it’s important to take the necessary time upfront to assess which method is the best to properly answer your research questions and objectives. Here’s a quick guide to some important factors to consider when determining whether or not Quantitative Research & Analysis is the right choice for your project; as well as when and how different quantitative methodologies are suitable, and some best practices for data analysis. Here we take a look at when to use quantitative research & analysis for a project.

WHEN should Quantitative Analysis be considered?

√ Client indicates they need feedback from more than a “handful of participants”.
√ You need to present a more convincing case to stakeholders.
√ Client desires to gather feedback from a diverse population of users NOT all located in the same place.
√ Client has limited budget (these approaches trim away things like travel and reduce the fielding hours, too).

WHAT are some key characteristics of Quantitative Analysis?

√ Quantitative Research is empirical in nature.
√ Observed and pragmatic.
√ It's conclusive in its purpose, as it tries to quantify a problem and understand how prevalent it is by looking for projectable results to a larger population.
√ Anwsers specific research questions.
√ Test hypotheses.
√ Uses measurable metrics to uncover patterns.

WHAT are the benefits of Quantitative Analysis?

√ Large sample sizes (typically 100+) make metrics captured statistically more significant, and projectable to a larger population.
√ Widespread participation (geographically diverse).
√ Conducted in the user's "natural" environment (e.g., home, office, school, etc.).
√ Often less research effort to complete (i.e., fewer hours than traditional in-lab, assuming it's a basic project).

WHAT are common Quantitative Methodologies?

√ Competitive Benchmarking
√ Quantitative True Intent Studies
√ Card Sort / Tree Test
√ Custom Quantitative Studies
√ Open Web Research
√ Surveys
√ Max-Diff

Quantitative Research & Analysis can be helpful at any stage

It’s important to pick the right methodologies for each stage. There are many different types of Quantitative Analysis. Select the one that’s best suited to answer your specific research questions and objectives.

Do you want know who’s visiting your site, why they are coming, what they like/dislike, and how well they succeed in accomplishing what they set out to do? Then, running a Quantitative True Intent study may help you to identify issues currently experienced by real users to provide insights for improving the design.

Or maybe you want to test some prototype designs to help refine your design prior to launching, then A/B testing is an effective way to test.

Virtual Card Sorts and Tree Tests are popular methods for understanding the mental model of the user. This can be the right method to test existing or proposed navigation/ site architecture.

Benchmarking studies at regular intervals (such as quarterly or annually) or before and after rolling out a redesign are helpful for measuring the impact site changes have on the user experience.

And of course, ongoing usability and benchmarking studies are appropriate for continually evaluating, optimizing, and maintaining your site or app.

What Best Practices should be followed when analyzing quantitative data?

#1 Rule: Work smarter, not harder!

√ No "ad-hoc" analyses.
√ Pre-planned analysis: Know what you will do with every piece of data you collect.
√ Quantitative analysis will help you confirm or debunk hypotheses.
√ Synthesize your findings into insights.
√ Run analysis of key performance indicators and metrics.
√ Include behavioral data and qualitative feedback to support the "why" behind the results.
√ Identify best-in-class examples.
√ Think through implications; make recommendations for next steps.

KEY LIME INTERACTIVE is a user research firm that conducts both qualitative and quantitative research for Fortune 500 companies, mid-to-large design agencies, and government agencies. We make it our business to answer research questions that help a company move the needle in terms of acquisition, convergence, and customer experience differentiation. For more information, visit our website at www.keylimeinteractive.com

Please send any questions/inquiries to marketing@keylimeinteractive.com