How to conduct efficient User Research?

Gwendoline, one of our Design Directors, recently contributed to the Product Inbox newsletter, edited by Timothé Frin, together with two UX Researchers. We thought this was a great topic, and all contributors kindly agreed to see the article translated for the Backstage blog. A big thank you to them all!

For this first special edition, I wanted to call on experts in user research to produce a comprehensive, concrete and actionable edition.

The aim was also to illustrate how different companies conduct user research, as there are so many methods and processes to explore.

A big thank you to my three great guests who produced all the content of this edition.

I am referring to:

Marion Damiens, Senior UX Researcher @Blablacar
Grégoire Devoucoux du Buysson, UX Researcher @Mozza & Cofounder of Cousto.io, a brand new training course dedicated to UX Research
Gwendoline Sonzogni-Jourdan, Design Director @PayFit

DISCLAIMER: for the same question, you will see that the answers of Marion, Gwendoline and Grégoire can be very different. This is due to several reasons:

There isn’t one single way to do user research
My 3 guests work in very different businesses (e.g.: marketplace for BlaBlacar vs. SaaS platform for PayFit)
My 3 guests work with different targets (e.g. B2C for BlaBlacar vs B2B for PayFit)

To produce this edition, I asked 10 questions to Gwendoline, Marion and Grégoire:

1. What makes good user research?

2. What are the mistakes to avoid?

3. Do you carry out user research alone?

4. How do you recruit and encourage participants?

5. How do you structure a script for conducting UX research?

6. Should the user be given autonomy over the product during the interview?

7. How do you moderate and lead the tests with the user?

8. What tools do you use to collect the data?

9. Where can I find models or templates to help me?

10. What resources would you recommend to go further with UX Research?

Key takeaways 🧠

1. UX research as a routine: user research is THE key to launching a product and making it last. It must be an integral part of company culture.

2. Prepare UX research: it must be part of the daily routine of product development actors, requires a solid framework and must not be rushed.

3. Involve the stakeholders: it must be shared by all the stakeholders of a project as early as possible. The interview phases should be carried out by several people to reinforce the involvement of all parties in the project.

4. Incentivise testers: an incentive in the form of a gift voucher or goodies helps to retain users/testers who give their valuable time. The relationship with them should be personalized.

5. Empowering testers: during a user test, the goal is to let the person navigate the product independently. This is more or less feasible depending on the prototype that is put in their hands

6. Reassure the testers: when conducting a user interview, keep as much neutrality as possible, to reassure the user. Drop your company’s branded mugs and t-shirts. Stay as quiet as possible.

LETS GO!

If you are new to user research, we will try to give you an overview, but above all to give you practical tips that can be applied in your daily work.

User research (or UX research 🤓) is a set of methodologies that allow the collection of quantitative and qualitative data. The aim of this collection is to understand and identify the needs of users of a service or product.

You are probably wondering when is the right time to carry out user research?

The answer is simple: almost all the time.

More concretely, user research is essential for launching a business, introducing new features, improving the user experience of a software…

In many innovative companies, user research has become a job in its own right.

The need to continuously improve the user experience and to make a product (or service) evolve according to changing needs.

It is the job of our three guests, who have made UX research their expertise.

For the rest of this edition, Gwendoline, Grégoire and Marion will answer the questions I asked them. The aim is to give you a concrete overview of UX research and how to use it effectively. I hope it will help you! :)

1. What is good user research?

Marion (BlaBlacar)

For me, good user research means not rushing into things without having structured your project a little.

At the very least, it is necessary to:

Set the context of the project, list the objectives, questions and hypotheses to be answered by interviewing the users.
Choose the most appropriate methodology according to the objectives (exploratory or evaluative?) and the context of the project (choose remote to reach a complicated target more easily; choose a tool without moderation if you are short on time)
Define your audience well and allow time to recruit the right users: sloppy recruitment can have a real impact on the quality of the feedback collected!

This article provides a complete template for asking the right questions when framing the research.

Finally, I would like to add that this is not a task that is carried out by the researcher alone in his corner.

You have to get the project team on board, both during the preparation phase (aligning everyone with the objectives; preparing the material) and during the fieldwork phase (allowing stakeholders to see the interviews/tests/focus groups live). Having involved and committed stakeholders makes it much easier to buy into the insights afterwards!

Gwendoline (PayFit)

It allows you to speak on behalf of real users. This is Catherine, Thomas and Sarah... You have met them, you know their story and you think about them in the design of your product.
It allows you to address your initial assumptions (read more 👉 Lean UX by Jeff Gothelf).
It is not a bottleneck, but a weekly practice. In my tribe, at PayFit, we organize ourselves to have at least one contact with users every week: it can be a user test, an interview, watching your users use the product (via Fullstory) or participating in product demos with the sales people.

Grégoire (Mozza & Cousto)

The most important thing is to frame the research questions on the basis of the decisions you want to make as a result. These research questions set the boundaries and provide direction. On this basis, the sample to be interviewed can then be defined and the interview guide written.
Secondly, recruitment is key because asking the right questions to the wrong people will not reveal insights that will help us make decisions.
Finally, beyond the necessary quality of the analysis, the last key is the restitution of the insights to the teams that will make the decisions and implement the solutions. Involving your teams early in the process and during the execution phase of user research simplifies insights’ delivery.

2. What are the mistakes to avoid?

Marion

Not defining clear objectives or listing the questions you want to ask
Choosing the wrong methodology, which will not answer the questions you are asking
Recruit the wrong people, because you didn’t have the time or didn’t think about the audience to interview
Not communicating about your research / not involving stakeholders enough

Gwendoline

Starting without having any hypotheses. The risk is to do a search that is too broad. It will be interesting, but you risk losing time.
Don't involve the engineers. Everyone needs to be in touch with the users and know Catherine, Thomas and Sarah, your real-life personas!
Seek to confirm your point of view. Stay open, listen and care for the users. It's like giving a gift, you want to make the other person happy, not yourself. And for that, you have to listen to the other person. Really. You don't want to know if you're right... you want to please them!
Not be silent. You are not there to convince the person in front of you. You are there to discover the limits of your assumptions!
Telling yourself that you are recording, thus not taking notes during the exchange. You will waste time listening to everything again and taking notes afterwards.
Wait several days before summarising your interview (you've forgotten everything, you don't understand your notes...)

Grégoire

Thinking that you can learn everything with a research phase and trying to know everything. On the contrary, you risk learning nothing solid and getting lost in the analysis.
Let the user research phase become a Christmas tree on which everyone adds their question. It is important to remain consistent with the initial framework.
Not identifying the limits of the methodology used. For example: asking open-ended questions without a response structure and without a context on a quantitative (numerical) sample. Or doing a quantitative analysis on a qualitative sample (e.g.: 9 out of 12 users plan their holidays between one and three months in advance. This data has no statistical value and can be dangerous for the analysis if it turns into 75%).

3. Do you carry out the user research alone?

Marion

At BlaBlaCar, user research is an integral part of product design.

There is a dedicated team that works closely with the PMs, designers, etc. We are currently 2 researchers and we work closely with them. There are currently 2 researchers and we split the squads to support all the research needs. We manage research projects from start to end, from the scoping phase to the fieldwork and analysis. The only part we don't handle is the recruitment of participants. This can be quite time-consuming and we are lucky enough to have a budget that allows us to outsource this part!

Gwendoline

There are no rules. It's a matter of team, motivation and ego. If you know that you tend to ask biased questions, leave it to a colleague less involved in the project and practice with internal users 💪.

For me, two people are the minimum to conduct an interview or test. There are two main reasons:

Alone, it's harder to ask the questions and take the notes at the same time. Some are very efficient at doing both, it's up to you 🤷‍♀️
The more team members that attend, the better experience. You will have the same repository during the design phase.

Because like many of you we at PayFit are working remotely these days, here’s how we do it:

Either we call the client by phone, and create a zoom with the team who are spectators (the user might be confused to see more than 2-3 heads in the zoom). At the end, we do a quick debriefing between PayFiter.
Or we record the exchange with the users (with their consent) so that the team can watch the interview offline (it's practical, it allows you to watch in fast motion 😉).

Grégoire

I never do the whole thing on my own because part of the team working on the project is at least involved in the framing and restitution phase. I try as much as possible to share the first interesting elements with them during the execution phase or to invite them to sessions.

Finally, I sometimes work in pairs with another UX Researcher, an exercise that I particularly enjoy, as it allows me to have another perspective than my own, to learn from our pair and to be able to give each other feedback.

4. How do you recruit and encourage participants?

Marion

Most of our user research projects involve our own members.

Depending on the project, we won't always want to interview the same profiles though.

Some projects will involve our drivers while others will only involve passengers (editor's note: Blablacar is a carpooling platform, linking drivers and passengers). And within these two segments, there are even more specific criteria.

For example, we are currently working on a project that will only concern new drivers who have already published a journey on the platform, but have not had a confirmed reservation for their journey. Once we have listed all the criteria, we then ask the data analysts to extract from our database people who correspond precisely to these criteria.

Then we work with a specialised company who contact the members and schedule the interviews for us! One important thing is that we only contact people who are opt-in in our database, i.e. people who have given their permission to receive communications from BlaBlaCar. Finally, I advise you to always provide an incentive to thank the participants for their participation. It is very important to show them that we are grateful for the time they have given us to help our project move forward!

We use Wedoogift but it could be a discount/voucher/promotional code linked to your product, or even goodies!

Gwendoline

It's always easier when you offer a gift voucher, discounts or goodies, especially if your product is still young. But it's avoidable:

In B2B, customers are usually more engaged and want your product to evolve. At PayFit, we contact customers who have opened tickets for the experience we are researching. They feel listened to and want to participate in improving the feature they had a problem with.
In BtoC, I relied heavily on users who were fans of our offer, they were my little community that I called upon very regularly. We did co-creation. And if not, I used agencies that took care of recruiting and rewarding participants (this is a significant time-saver when the team is small).
And if your product team has the resources for it, having a community of users willing to improve the product makes recruiting much easier. At PayFit, every quarter, the design studio assigns a dozen or so members to each squad who are interested in their functional scope. We save a lot of time on projects!

Finally, to retain your testers, a good practice is to value them: make videos with them, invite them to meet your team, offer them an exchange over a snack, travel to meet them if it is easier for them. And if they are motivated, organise co-creation workshops with them. At PayFit, in a few weeks we will be organising one on navigation!

Grégoire

Recruitment is a key step and rarely simple.

To contact and encourage participants, the following points seem important to me:

Create a personal contact with each participant, even if it's an automatic campaign you have to find a way to show that you want to talk to them for X reason. Working with a growth manager can be a good solution.
The participant should feel important and special. Example: "you are one of the 10 selected profiles because you have done X".
Offer a clear incentive even if it is not monetary.
Adapt to the participants (e.g. let them choose the video conference tool they prefer) rather than asking the participants to adapt to us.

5. How to structure a script to conduct UX research?

Marion

Whatever the methodology, it’s always the same process to follow when writing a protocol/script/questionnaire/interview guide. In fact, there are many different titles for this document which has the same purpose. This script consists of the following steps:

A warm-up to put the participant at ease, explain the context of the interview, reassure them: "we are not testing you".
Introductory questions: ask more general questions to better understand the user's profile. Then you can adjust the scenario you are going to ask them to carry out.
Depending on the methodology: there will then be exploratory questions (using the funnel principle, going from the broadest to the most specific subject) or a test scenario (for the user test) with questions.
Finally, concluding questions, to make an assessment

Gwendoline

Before the script, I would like to share with you a framework for organising user tests. Here is a diagram from The design thinking toolbox that might interest you:

Then concerning the script, there are principles but no models.

Here is an example of a test structure when you plan a minor evolution of a feature (and you didn't have time to do a good discovery before the project ... I know it can happen 😉 ):

Ask the user to share his screen and show you how he does today to accomplish the action you are interested in.

Encourage them to formulate their current pain points. Beware that sometimes the user may be very attached to the current way of working and find it perfect.

Have them test the prototype to get them to perform the same action.
At the end, you can invite your user to compare the two.

Grégoire

Important principles for writing the guide and conducting an interview:

Start by checking and digging into the participant's profile.
Start with a simple topic/experience before going deeper.
Ask for concrete examples and help the participant to activate his or her memory by going back through the story together. For example, you can use a tool like Jamboard.
Avoid using conditional tense in questions and answers.
Do not talk about it being an interview but rather a discussion.
Do not hesitate to go back on an answer if you think you have not gone into enough detail.
Avoid vocabulary that invalidates (e.g. "Why do you think that?") or confirms ("I agree with you") the answer to your question. Use neutral vocabulary ("I understand").

An example of an interview guide here.

6. Should the user be allowed to be autonomous on the product during the interview?

Marion

Ideally, yes, but it's not always possible.

It will really depend on the equipment you have available for the user test.

For example, on a live product, the user is in total autonomy, the only limits will be those that you set in the scenario (e.g.: "I'm going to ask you to create an account").

But on a prototype, it gets more complicated. Depending on the level of fidelity and the possible interaction on your prototype, you can leave more or less autonomy to the user.

Basically, the less functional the prototype is, the more likely the moderator is to intervene ("if you click here, nothing happens because you are on the prototype").

At BlaBlaCar we have chosen to have high fidelity prototypes. This allows us to eliminate the bias of having to constantly re-route the user on the prototype.

Gwendoline

It's the best test in the world. But you need a working prototype that allows it.

As a facilitator, you only have to fill in the gaps with explanations of what your prototype can’t do.

I once had the need to do a test with very limited wireframes. That was enough to go and test my assumptions. In this case I don't even give the user the lead. I show them an interface and ask them my questions such as:

What do you understand on this screen?
What do you imagine is behind this button/link?

Grégoire

The idea is to get as close as possible to reality while taking into account the context of the test (prototype, device, screen sharing, etc.) and using it to dig into what is going on in the participant's head at each stage: what questions does the user ask himself, what does it evoke in him, etc.

7. How to moderate and lead the tests with the user?

Marion

Making the participant comfortable is really important so that they feel confident enough to share their opinion and point of view: reassure them about the nature of the exercise, explain that it's not them that we're going to judge but the interface, and tell them (if this is the case) that you haven't worked directly on what you're going to discuss or present to them.

Then you can use all the active listening techniques: show the person that you are listening, rephrase to make sure you have understood correctly, don't judge or react to what the person is saying: remain neutral as much as possible, even if it's not so easy! And of course, do not hesitate to ask additional questions if the answer was not clear and/or precise enough.

In his book "Rocket Surgery Made Easy", Steve Krug compares the role of the moderator in user research to that of a therapist, and it is strikingly true!

Gwendoline

I would encourage you to develop complicity with your user. They will remember it well and it will give you access to moments of confidence that will help you understand their real needs.

Then, I have two well-known tips for designers:

In the introduction, ask your user to think out loud. If he stops doing it (because it's not very natural), don't hesitate to ask him again.
When your user is blocked, don't help them. At least not until you have asked them what they think should happen next.

Grégoire

To run a user test, the first thing to do is to set up an environment that is as neutral as possible (forget the company T-shirt, stickers or corporate mug) and to make your user's life easier: for a video-conference, remember to clearly indicate all the steps to get to the call.
Put the user at ease with 1-2 mins of small talk.
Avoid creating a hierarchy between you and the user or the prototype being tested and the user. If the user feels that a team of 10 people have worked for months to achieve this result it will be more difficult for them to express themselves freely.
Keep your mouth shut, observe, let the user struggle if necessary and then come back together to dig into what happened.

In this podcast (from 13'), Michael Margolis (UX Researcher at Google Ventures) shares his experience on posture and questions to ask.

8. What tools should you use to collect data?

Marion

Lookback, which allows you to set up interviews/tests in a very functional way (you can see the participant's face, screen and navigation)
Maze, which allows you to set up unmoderated tests
Survey Monkey for quantitative questionnaires
Notion to centralise feedback & insights (tagging system)

Gwendoline

Our User Researcher, Mathilde, has set up Dovetail at PayFit.

It allows each team to put their interview summaries on it and to tag them with common tags to the whole product team as well as tags dedicated to the project.

It's a great tool for 2 reasons:

When you have finished your research and tagged all your interview summaries, you have a table that allows you to generate insights super easily. It's wonderfully practical!
For the other teams, when they start their discoveries, they can look at the knowledge accumulated on their subject through all the interviews that have been done. The more history there is, the richer our hypotheses are at the beginning of the project.

Grégoire

Note taking and qualitative analysis: Notion, Condens
Video interview and user test: Whereby, Lookback, FB Messenger, Zoom
Quantitative survey: pollandroll, qualtrics, typeform

9. Where can I find templates to help me?

Marion Damiens

The Usability.gov site shares a lot of UX Research templates at all stages of the project: it's great!

Gwendoline

Ideo, a company known for democratising design thinking, has created a design kit that can give you practical tools to do user research!

Grégoire

Google Ventures shares various simple templates for start-ups who want to start UX Research.

Miro also shares many templates created by different teams on Miroverse.

Marion

The "Quote" podcast by Roxanne Lacote (in French) and Awkward silences

The book UX Design Methods by Carine Lallemand. Great for beginners!

Gwendoline

🎧 Mathilde Gauthier, User researcher at PayFit, shares in this podcast how we grew the User Research in the Product team at PayFit.

🎧 Chloé Martinot (ex Mano-Mano) gives practical tips on how to do research in this podcast. Great for digging into what we've covered here.

📖 Testing business idea (by Bland and Osterwalder) gives a lot of tools to test your ideas with users.

🖥 NN Group blog, Research method section: the articles are accessible and educational!

Grégoire

3 books:

Thinking fast and slow, Daniel Khaneman

Just Enough Research, Erika Hall

Research practice, Gregg Bernstein

3 podcasts:

Mixed methods

Design Journeys (Design, Research) by Gauthier Zimmermann

Quote by Roxane Lacotte

Thank to Timothée Frin and all contributors for allowing us to translate this piece and to publish it on Backstage!

Key takeaways 🧠

LETS GO!

1. What is good user research?

2. What are the mistakes to avoid?

3. Do you carry out the user research alone?

4. How do you recruit and encourage participants?

5. How to structure a script to conduct UX research?

6. Should the user be allowed to be autonomous on the product during the interview?

7. How to moderate and lead the tests with the user?

8. What tools should you use to collect data?

9. Where can I find templates to help me?

10. What resources would you recommend to deepen UX Research?