Strengths and Weaknesses of a True Intent Study

strengths-and-weaknesses-of-a-true-intent-study

Do you really know your users? Usability testing is great way to figure out how someone would interact with your app or site, but the results are only valid if the people you bring are representative of your real users. The True Intent methodology is one of, if not the best UX approaches for learning who your users are, what they intend to do on your site, and how successful they are in doing that. Let’s explore exactly what is this particular methodology, and focus on the strengths and weaknesses of a true intent study.

1 ) IMPLEMENTATION- HOW WILL YOU RUN THE STUDY?

A Tue Intent study requires selecting a an online software tool to script the survey, and to insert that survey script on your website. Selecting that tool is important because each tool has its own strengths and weaknesses that aren’t always apparent from the company’s website. We have a guide to selecting a quantitative research tool available here, but we’re happy to talk you through it as well if you’d prefer; you can find our contact details at the end of this article. Keep in mind that you’ll need to work with your website development team to insert a single line of javascript on your site code in order to make this work, among other things. This may impact your timeline for research depending on how your company handles changes to the site’s code.

2) INFORMATION GATHERING

Next, you need to determine what to ask test participants. The biggest value of a True Intent study is that they capture the organic population of site visitors, therefore it’s a perfecto opportunity to find out who your users are and what their goals upon visiting.

Typically these studies begin by capturing basic demographic details such as age, gender, household income, and occupation to give the researcher a clear picture of who is visiting the site. Additionally, this is great opportunity to capture how the user came to your site. Was it from Google? Did an email campaign spark sudden interest? Were they referred by a friend?

All of these dimensions are important in understanding how to best serve your users; in fact, if you’re building behavioral personas to illustrate your user base, this is an excellent way to begin.

The collected data is rich and authentic, these are your real customers, in their natural environment, doing something they were going to do already before you intercepted them on the website. Contrast this with an in-lab usability study where you have called someone into an unfamiliar room, sat them in front of a camera and/or one-way glass, and asked them to perform a specific task.

It’s clear that the True Intent study would give a more natural, honest and unadulterated opinion from your customer.

3 ) LEARN ABOUT THE PARTICIPANTS EXPERIENCE

Next in your survey, you’ll ask the user what they were hoping to accomplish on your site. Prior research will inform some discrete answer options for this choice to make your data analysis easier. After the user answers this question, the survey drops out of focus and allows the user to continue the task they were already performing. Once they’ve completed that task or closed your website, the survey comes back into focus and asks follow up questions about the experience:

Was the customer successful in doing what they came to do? Why or why not?
How satisfied is the customer with their experience?
How likely would the customer be to recommend this website to a friend? (NPS)

Most survey tools will also capture metrics like the amount of time spent on the website, or even what pages were accessed during the visit. The combined data from the metrics and the questions above will help you identify any nagging issues with particular tasks on your site. Measuring satisfaction, and other metrics such as how easy information was to find, can provide a measuring point for the current state of your site that you can reference later after site changes were made.

One of my favorite projects was a True Intent we ran on a quarterly basis for a large Telecommunications provider. We had intercepts set up at multiple pages on the website to identify the needs of customers who entered on different pages and target areas for improvement. As we identified usability issues and customer profiles, our client was able to make changes to the pages and see the satisfaction and Net Promoter Score increase over time. As a usability professional, getting a live feed of the effects of your research recommendations is one of the most satisfying parts of the job.

At this point, you may be excited about this methodology, and ready to fire this up on your website and solve all of your problems, right? Well hold on one second, because we haven’t talked about the weaknesses or hindrances of True Intent studies.

THE QUIRKS

The nature of True Intent lends itself to some quirks. One thing to keep in mind is that the data that you get is dependent on your site traffic. If you try to put a True Intent intercept on a page that doesn’t get many visitors, you may never get a significant sample size for a long time. You also run the risk of annoying site visitors with the intercept pop-up.

Additionally, like any sampling method, the nature of True Intent can lead to a response bias, where only the least satisfied users who have complaints to get off their chest will actually take the survey. Additionally, if site traffic is slow, the intercept doesn’t lend itself well to incentivizing participants. Word can spread quickly that a site is rewarding participants, and you’ll start to get visitors coming just for the reward, who wouldn’t be a user otherwise.

Finally, the software required can create complications in some cases, as I mentioned above. First, you have to license the software for your study, which adds to your overall cost. Second, you need to work with engineers or web developers to get the code out to your website. This javascript code is small and easy to implement in most cases, but has delayed studies in the past due to scheduled website rollout periods.

It’s important to weigh the positives and negatives of any methodology before moving forward, and True Intent is no exception. Keep the above factors in mind when making your decision, and you can execute your research fully confident in what you’ll receive. At Key Lime Interactive, we have team members who have performed True Intent studies for over 10 years, and helped develop and maintain some of the original tools for performing this type of study. We would love to speak with you regarding your website needs and whether True Intent is the right methodology to achieve your research goals. Contact sales@keylimeinteractive.com to find out more about how we can make this solution work for you.