5 Commonly Used Metrics in User Research

Written by Key Lime Interactive | Nov 10, 2016 4:26:09 PM

The only way to assess whether a product enables users to achieve their intended goals is by measurement. Metrics are standardized methods of measuring aspects of user experience to establish benchmarks, and evaluate design interactions over time. Here are 5 commonly used metrics in user research explained:

5 User Research Metrics

Task Completion measures, by observing simulated or actual use, whether or not the user completed the task successfully. Whether most users are able to successfully complete tasks central to the product or service is a strong indicator of overall usability. At the simplest level it is reported as either pass or fail, but it is also often useful to report a third category, succeeded with difficulty. Reporting success with difficulty captures observed frustration, near-misses, and moments of confusion that ultimately resulted in successful use. Patterns of observed difficulty highlight usability issues that may not result in outright failure, but can cumulatively have a large impact on the overall user perception and use of the product. Typically, task completion is reported using the percent of users (e.g. 83%) who were successful at fully completing the task (arrived at correct page).
Time on Task is a related efficiency metric that records how long it took the user to complete a task. Typically measured against an estimated or average score, lengthy completion times are correlated with usability issues. Sometimes number of clicks is also used to measure efficiency for software or an app navigated by a mouse or by touch, but this can also result in inexperienced designers solving the problem by overloading a single screen or menu with options and creating information overload.
System Usability Scale (SUS) was developed by John Brooke, Ph.D., in 1986 as an inexpensive and fast way to evaluate any software or hardware system. Users rate their degree of agreement with ten statements about ease of use, using a five-point Likert scale (pronounced "Lickert"). One representative statement is "I think I would like to use this system frequently." The individual scores are calculated resulting in a final score on a 100-point scale as a measure of overall user satisfaction and their perception of how easy it is to use the product. Now celebrating its 30th anniversary, the SUS has proven to be a robust, valid measurement tool that has become an industry standard.
Single Ease Question (SEQ) is an ease-of-use metric designed for evaluating individual tasks, rather than entire systems like the SUS. After a task, the user is asked to rate, on a seven-point Likert scale, how easy or difficult they found the particular task, and often asked to explain their rating if they found it at all difficult. Since ease-of-use and user satisfaction are strongly correlated, but not identical with usability, this question is often used in combination with other metrics to assess overall usability. It is typically reported in a top-two-box fashion, where only the proportions of the top two positive response options are reported. While this is fine for a stand-alone measurement, for the highest precision and tracking variability in user responses to the new product or interface iterations over time, it is best to use the entire scale.
Net Promoter Score (NPS) measures user loyalty by assessing how likely they are to recommend a product to a friend or colleague. Users respond on an 11 point scale, where 0 = not at all likely and 10 = extremely likely. Those responding with 9 or 10 are considered "promoters", 7-8 are "passives" and 0-6 are "detractors." Promoters are the ones considered the most likely to spread positive word of mouth, passives are neutral, and detractors are customers that are likely actively negative and potentially discouraging others from using the product. A final score is an absolute number between -100 and 100, which is the difference between the percentage of promoters and detractors.

These five metrics provide a way to measure the overall user experience of a product, but none are sufficient by themselves. Measurements enable researchers to quantify the experience of using a product, but knowing what happened does not explain why it happened. To truly have a holistic picture of the user experience of a product, skilled user researchers combine appropriate metrics with qualitative and contextual inquiry to develop a deep understanding of the root causes behind issues affecting usability and user satisfaction.

View full post