Dos and Don'ts of Diary Studies

A diary study is a research method that consists of collecting data about user activities, behaviors, and feelings over a certain time period, ranging from a few days to a month or more. During a diary study, participants will self-report their experiences and activities on a diary (or digital diary such as Dscout, Experience Fellow, Revelation, or just Google Form). Participants will self report their interactions, thoughts, and feelings in regards to a product or organization. Diary studies are useful to understand long-term behaviors and potentially create Customer Journey Maps through being able to document the customer's interactions with the product or an organization.

Let’s analyze the Dos and Don’ts of a Diary Study.

Do's of the Study Setup

Use the right tool
Depending on the scope and the amount of data that you are planning to collect, you can pick a tool that is more robust and that requires participants to create an account (like Experience Fellow, Dscout and Revelation), or you can pick a familiar-to-most tool like Google Forms or Typeform that doesn’t require your participants to be necessarily tech-savvy.

On-board your participants
During the onboarding you will have the chance to meet you, participants, build rapport, and answer any questions they might have. Use this meeting to convey the importance of the diary study they are participating in.

Set expectations
Make sure that participants know what will be required of them and have a clear and defined timeframe to complete their activities.

Collect the right information
Use a mix of qualitative and quantitative questions.
- Your quantitative data will help you understand success/failure rates and satisfaction ratings.
- Your qualitative data will help you better understand what the reasons behind those success /failure rates and satisfaction ratings are.

Do's of Data Collection:

Provide updates
Update clients and stakeholders regularly regarding the progress of the study. Include part of the initial findings in the update will go a long way with making the client feel involved in the study.

Set up reminders
Most tools allow you to set up daily reminders and alerts to participants that did not complete their daily assignments.

Track participants progress
In order to stay on top of participants progress, spend about 30 minutes a day to monitor their entries. If a participant didn’t complete his daily assignment send him a reminder.

Be ready to troubleshoot issues
Both technical and personal issues with your participants will occur, you need to be

Conduct Debrief sessions with your participants
During the debrief you can ask questions that were not asked or
answered during the study.

Don’ts

Don’t make assumptions
Don’t assume your participants will dedicate as much time to the study as you think they will. Reach out to these participants and encourage them to complete late activities when they can.

Don’t ask too many questions
Consider reducing the number of questions and activities as participants progress towards the end of the study (it’s common in long studies for participants to get fatigued).

Don't forget to over-recruit
It’s very common for studies to have a few people that drop out. Over-recruit 20% of your study number (i.e. if you need 10 participants, recruit 12).

Don’t use always the same format questions
Variety can help keep things fresh and interesting for users. Keep a good mix of open and closed-ended questions (the more open-ended question, the more time the participants will have to spend and commit to the study).

Examples of some effective prompts:
- One word to describe your experience
- Upload a video or a picture that represent your interaction with the product
- Describe a day with the product
- Describe a day without the product
- On a scale from 1 to 5 how would you rate….

After the Diary study data collection

Once the data collection period is over, plan a follow-up conversation with the participant and discuss the specific details of the study.

You’ll be able to clarify any ambiguous details and ask additional questions to gather qualitative feedback on specific moments of the experience. You can also ask for feedback from the user to improve the process for future participants.

Nielsen Norman group recommends to “take a deep breath” before diving into the vast amount of qualitative data and think about the following questions when evaluating their behaviors: how do they evolve and change over time? What influences them?

Then build a customer journey map to help your client understand the entire user experience from the perspective of your participant.