What Is Bias, And Why Does It Matter In UX Research?

We all have biases.

What does this statement mean? In this blog post, I take some time to zoom out to consider the bigger context for this statement. First, it helps to think about the enormous amount of information available to us in any given second and to consider what we focus our attention on. Second, it is valuable to consider how this attentional focus and our personal experiences can influence what information captures or evades our attention. Finally, I offer some examples of why considering bias is important in User Experience Research (UXR). 

Prioritizing relevant information and ignoring irrelevant information

Before digging into bias per se, I think it’s worth taking a few steps back to consider the substantial amount of information we are exposed to. There is so much information available to us at any given second! We’ve learned to pay attention to some of this information and ignore other information – there’s just no possible way to consciously be aware of all the information we’re exposed to! For example, when we’re talking with someone, we tend to pay attention to the words being said or the gestures someone is making as they emphasize a point. Even when focusing on the words of a conversation, we tend to focus our attention more on content words such as tomorrow or pain point. We likely don’t notice the number of times articles such as the or an are used during a conversation, because most of the time, these words are not particularly critical to a conversation. Similarly, even though we may be face-to-face with a person and see their entire outfit, we might not pay attention to what color their shoelaces are or how many buttons are on their shirt. If we don’t pay attention to information, we miss it – even if we’re looking directly at it! Sometimes, we can anticipate what we might be exposed to. Maybe your friend is excited about the new color shoelaces they just got and mention their excitement to you. The next time you see your friend, you might remember this information and purposely look at their shoelaces. This one-time focusing of attention is not bias though. However, if you start looking at people’s shoelaces to draw conclusions about how cool they are (e.g., people who wear colored or sparkly shoelaces like your friend are cooler than people who wear the regular shoelaces that come with shoes), then you could say you have a bias toward people who wear different shoelaces (or, conversely, a bias against people who wear regular shoelaces).

As an example of how we naturally focus our attention, watch this short 2-minute video of a conversation. When people watch this video without any context or instructions, they typically focus their attention on understanding and remembering the conversation, because their past experiences of social interactions and conversations guide and shape what information they expect to be important. Often, this focus on the conversation is valuable! However, as shown in the video, by focusing on the conversation, we might miss other things that happen because our attention is focused there. However, if you watch this video (as someone who anticipates there might be something unusual outside of the conversation), you are likely to direct your attention to other aspects of the conversation beyond the words because you expect something else to happen so you watch or listen for it and therefore notice at least some of the editing errors.

Here’s another short 2-minute video to test your awareness: typically people focus on the commentary and miss many of the changes that occur in the background. I like this video because it’s a great example of how we do not notice or remember things that we do not attend to. We have to prioritize what information we pay attention to because there is so much information consistently available. However, by prioritizing some information, we may not even be aware of other information or things occurring. Why would you notice or pay attention to the color of the clock laying on the floor or what’s in a pot or what type of flowers are sitting off to the side in the background? However, if you collect antique clocks or are a master gardener, this information may capture your attention immediately – even if no one else notices it – because it is important to you, so it’s information you notice. This prioritization of some information and lack of awareness of other information is important when it comes to bias.

As an aside, the phenomenon of not noticing changes in subsequent scenes is known as change blindness and not noticing unexpected things in a scene is known as inattentional blindness; check out this video for more information. In many - though certainly not all -  cases, we may not even notice something happening that is having an impact on someone else because the information hasn’t risen to a level to capture our attention. It can be challenging to be aware of your bias if you are not aware that something else is happening. In order to become aware of our bias, we often need something to occur to help us notice the information we are not focusing our attention on. Want one more example? Check out this video. Be sure to follow the instructions along the way and actually try to get an accurate count of the number of jumps!

In UXR, we may find ourselves supporting a team member by taking notes during a session or assisting with data analysis. Given that each of us may focus our attention on different parts of an experience, it is important to understand what the research question is and the nuances of what the objective of the research is. This understanding can help you focus your attention on the key information needed to meet the research objectives. In addition, it is possible you might notice additional things given your fresh perspective and the information you focus on.

Personal experiences influence what we notice

Sometimes, we shift our attention to notice changes that happen in the environment. For example, if someone starts laughing in a quiet room, people are likely to shift their attention to the person to see why they’re laughing. We also shift our attention to changes in the environment such as the heat turning on, or the wind starting to knock a branch against the window. But as the heater continues running or the branch continues to tap against the window, our awareness of these events may shift again. Sometimes we habituate to the information which means that we get used to it and gradually stop noticing or responding to the information. For example, after a bit of time, we may not consciously notice that the heater is running in the background. In other cases, we may become sensitized to the information and actually become more aware of the information. For example, a branch continuing to tap against a window may become increasingly more irritating over time. Habituation and sensitization are two ways that that environment can impact our attention.

In UXR, you may start to habituate after several sessions if participants have similar experiences. Be careful in these situations that you’re not overlooking important information! In contrast, if a surprising point continues to arise session after session, you may become sensitized to this information and notice it more as the sessions continue.

One of the key things behind the statement “we all have bias” is that our individual life experiences influence where we focus our attention. Sometimes our attention is drawn to obvious differences between ourselves and others. For example, if someone who is really tall is in a room with people who are all quite short, the tall person may pay more attention to their tall height. If this experience of being the only tall person in a room of shorter people is a one time occurrence, then the tall person may not really think about their height again in other situations. However, if the tall person is regularly in the company of people who are all much shorter, they may start noticing height in more situations, i.e., have a bias to notice height. Moreover, they may start making an evaluation about this height difference they started noticing (e.g., I feel uncomfortable being the only tall person – I don’t like it) or even start acting differently (e.g., I don’t want to help reach the high things, so I will avoid eye contact). 

You might have noticed this change in what information you notice occurring as you gained more experience in UXR or even become more comfortable running a particular study after the first couple participants. For example, in your early sessions of moderating, you might have your attention focused on cueing up the stimuli, making sure the technology is all working so your stakeholders can observe the sessions, hitting all the core questions, ensuring you’re not asking leading questions, taking notes, evaluating success/failure on the task, and monitoring the time so you don’t run over. Whew! That’s a lot of places for your attention to be focused! So you might miss the participant’s confused face or hesitation before clicking which may have been a key time to prompt for more information about their experience. However, after a few sessions when things are running more smoothly and you’re more comfortable with the moderation guide, your attention is freed up to notice more. Although we have to prioritize what information we pay attention to, there’s a cost in missing other information.

Attention (or lack of attention!), bias, and UXR

So, at the simplest level, the saying “everyone has bias” can simply acknowledge that each one of us has our attention pulled toward some information/events and that some others are missed even when we’re directly observing it. However, when people say “we all have bias,” there’s often an underlying message that the bias is causing harm or has the potential to cause harm, even if it’s unintentional, or if we’re not aware of it. If we’re not aware of the biases we have – even at the simplest level of what information we pay attention to or overlook – it is easy to miss critical information in our research. For example, because of our bias, we may not leave space or may plan too much space in research sessions for some pain points to surface. When taking notes or analyzing the data, our bias could direct our attention to overlook or focus on particular aspects of someone’s experience. 

As UX researchers, we’re in a position of testing how people experience things. Being aware of our biases and what aspects of participants’ feedback we may focus on - or overlook - is important. It’s critical for both the research team and the participant sample to be diverse in order to capture experiences that may be missed or not occur in a homogenous team or sample. In addition, it’s important to keep reading and learning to become aware of the biases you have so you can consciously shift your attention to notice things you were previously unaware of.

In addition to the examples offered throughout this post, let me share a few high-impact examples of how racial bias negatively impacts products. 

• The technology for film and photos were largely developed based on white people. As a result, people of color often are not properly captured in film or photos. See this NY Times article or this Calgary Journal article for more information. Including people of color in the R&D process would have uncovered the need for further development. 
• Film isn’t the only technology not correctly recognizing people of color. Automatic soap dispensers, sinks, and facial recognition software are just a few examples of technology that could have been effectively developed for a wider range of people if a more diverse group were involved in the various R&D stages. UXR could have helped uncover these important limitations in usability tests.
• Bias negatively impacts technology in the health field too. For example, race corrections are programmed into medical devices such as the spirometry to measure lung function or equations to assess kidney function. This race correction makes it more difficult to detect health concerns or disease in Black people.
• For a more in-depth dive, watch the movie, Coded Bias which is a powerful look at how biases are coded into algorithms that reinforce discriminatory practices. Check out the trailer here!

I encourage you to identify other experiences and share them in the comments below. It may seem overwhelming to consider ways to avoid bias having extreme negative consequences. But you can start by recognizing your own bias. What information are you missing that’s occurring right in front of you? Are you recruiting a diverse sample of participants for your research? 

At Key Lime Interactive, as a woman and minority-owned business, we consciously strive to hire diverse team members. In addition, we have developed the Inclusivity Index^TM as a concrete starting point for raising awareness about who is recruited for studies. The Inclusivity Index can be a first step in becoming aware of whether your sample is representing various perspectives and experiences that may be over or underrepresented in research. Contact us for more information!