Considerations for remote usability testing

Preparing for remote usability testing

What you need

Let’s begin with what’s needed to get started running a usability test remotely:

A computer

The type of device isn’t important here. However, it’s worth noting that during these sessions the participant will be sharing their screen, so make sure your screen size is large enough to view what they’re seeing. Even better, two monitors will make your life a lot easier; one for notes, one to see the participant.

Video sharing software

What software should you choose to host the sessions on? Consider what’s needed, e.g., will you have observers? Will you need to see the participants' screen? Will you need live transcription or even translation services? There are many options to choose from, although what's best for you may not be best for your participants or clients.

A couple of tools to consider: lookback.io provides tailored solutions to separate participants from observers during sessions, although setup instructions can be confusing. Or you may be more familiar with Teams or Zoom. Zoom provides everything you need including screen-sharing capabilities and even live transcript for those who might need additional aid.

Microphone

In most cases your laptop microphone will do just fine, but investing in a USB microphone or headset is well worth it to make sure participants can hear you clearly.

Internet connection

A good internet connection is a must for the moderator of the session. Participants often turn up with a poor connection themselves, so making sure everything's up to speed on your side helps minimise any issues.

Recruitment

Regardless of whether we're recruiting participants ourselves or handing that task to an external party, there are a few points that need to be made extra clear to make sure the right participants are chosen.

Specify the required device

Making it clear at the initial point of contact that the session MUST take place on ‘x’ device should help prevent sessions being cancelled due to participants turning up on the wrong device.

Internet connection

There's never a guarantee when it comes to an Internet connection, especially with participants dialing in from their own homes. We can go some way to help ourselves by specifying a minimum connection speed when recruiting. A 5Mb download speed is usually suitable. If participants aren’t sure how to check this, speedtest.net provides a service to check connection speeds.

Web Browser

When testing a website or prototype that runs on the web, make sure to specify during recruitment what web browser is required. Also, what version of this browser's needed. This'll avoid participants using out of date browsers that are incompatible.

Technical ability

Last but certainly not least is technical ability. You want to test your products with people who have all kinds of experience using computers. However, especially for remote sessions, if a participant is not able to use their device well enough, you risk losing valuable testing time helping them maneuver your chosen platforms.

Pre-session

Usability sessions can be daunting for participants, so it’s important to make everyone feel welcome. Make yourself known by sending an invitation email to the session, introduce yourself! This is best done roughly a day or two before the session takes place.

This email is also a great time to re-confirm the device that the participant will be using in the session, and the time for them to arrive. It’s always a good idea to ask participants to join the session 5 minutes early to get over any possible tech issues while connecting to the call and getting screen sharing up and running.

During the session

Session intro and welcome

The start of a session plays a vital role in setting the tone and allowing your participant to feel comfortable and at ease. The nature of moderating remotely means that there has been no initial greeting as they arrive at the testing venue. This can lead to the participant being thrown right into the session when they join the call, which can be overwhelming.

To get the most value from your research, don’t jump right into the intro, but instead take a minute to chat. Perhaps ask how their day or week has been so far. Just something simple to engage in a non-scripted conversation to allow everyone to settle in. Then walk them through what's going to happen during your session.

Moderating the session

The makeup of the usability test itself shouldn’t be much different than if it was conducted in person. However, there are a few things to adapt when moderating remotely:

Speak less

Video software such as Zoom does a really great job of cutting out background noise and muting microphones when someone is talking. However, when it comes to your moderation, giving regular audible clues to show you're listening such as ‘Okay’ and ‘Yes’, can cut out the participant’s audio mid-sentence for a few seconds before it picks back up.

During testing it’s vital that the participant is heard, and although unintentional, you can interrupt their dialogue and lose insight purely by speaking/making a noise. It's good practice to speak as little as possible anyway as the moderator, but even more so remotely! Replace audible cues with more visual ones where necessary.

Observers

As mentioned earlier there are tools, such as lookback.io that provide a service to separate the participant and observers, much like having an observation room behind glass if the session was in person.

However, if you’re using software like Zoom, participants will be able to see others on the call. This means there are some rules you should follow to not disrupt the session, and to make sure the participant feels comfortable:

• Make sure there's someone on your team whose role during the session is to write notes. Taking this pressure off the moderator allows them to be more engaged with the participant, asking follow-up questions rather than making sure every insight is captured.
• Limit observers to a max of 5 per session. Allowing more than 5 can become intimidating for a participant and may start to affect the outcomes of the session. It’s great to have others watching to provide additional insight and analysis, but keeping these numbers limited is respectful towards the participant.
• All observers should arrive at the session before the participant, keeping their cameras off and microphones on mute. This is a must!
• It’s great practice to allow observes to provide additional probes or questions for the participant during the sessions, but these should always be asked by the moderator to not have observers speaking up mid-session. Agree a method of communication between the moderator and observers before the testing so that everyone is aware how to ask questions without it being visible to the participant.

Conclusion

Although the objectives of running testing remotely and in person are the same, following the above tips will help your sessions run smoothly and maximise the value and insight gained throughout the project.

There's no one correct way to conduct user research, and each different project or product being tested will almost certainly mean adapting how you approach each session or focus group. However, one objective that should be constantly at the forefront of your project goals is to build products and services that meet the needs of real users, presented in a way that aligns with their natural behaviour.

Take a look at our Discovery and UX services to see where we can help.