It’s important to understand how people experience digital content on different devices in any situation. Find out how the Digital Public Service team carried out remote usability testing of government contact information on mobile devices during the COVID-19 pandemic, and what they learned along the way.
Why we focused on contact information
Govt.nz is often the first point of access for people looking for government information and services. The site is responsible for hundreds of thousands of referrals to government organisations every quarter.
As part of the Govt.nz vision to empower access to government, we want to make sure it’s easy for people to find contact information for government organisations and services, no matter what device they’re using.
In 2021, Govt.nz received over 2,400 emails that were intended for other government organisations.
The time and effort involved in processing emails intended for other organisations and connecting people to the right place, can result in unwanted delays, create a poor experience for people seeking government information, and put additional pressure on the Govt.nz support team.
So we initiated research to check if the current design and location of government contact information on Govt.nz was causing users to contact the Govt.nz support team with queries intended for other organisations.
Conducting usability testing in a pandemic
We know that more people are accessing Govt.nz on mobile devices. In 2021, 55.84% of visits to Govt.nz were on mobile versus 41.50% on desktop computers.
So we wanted to see first-hand how people interact with Govt.nz on mobile to identify how we could improve their experience and findability of contact information.
Challenges
Most of our qualitative usability testing has been done in person. Because of pandemic restrictions we needed to carry out usability testing remotely.
Remote-moderated research on mobile was new to us. We looked at how others did it and discovered we didn’t have the recommended equipment — so we took advantage of the tools we had available.
What we did
We knew that many people use cloud-based video conferencing platforms like Zoom on their mobile devices, so we chose to use Zoom for our testing.
The benefits of using Zoom are that:
- it allows for users to share their screens
- there’s no cost to participants
- it’s approved for use by the Department of Internal Affairs.
You could try any video conferencing platform that suits your organisation.
We gave participants a couple of tasks intended to prompt them to look for contact information and monitored their interactions, behaviour and comments as they were going through the website.
What we learned
Prepare
- Practise running the sessions on different mobile operating systems, such as iOS and Android to be aware of any differences.
- Check your participants have a compatible mobile device and that Zoom is installed.
- Give participants instructions to download and use Zoom a few days before the session.
- Ask participants to test their audio and video prior to the session.
- Build in more time to check for technical issues and to walk people through how screen sharing works.
- Follow ethical research practices — provide safe procedures, seek informed consent, store personal data safely.
Consider privacy
You are testing on someone’s personal device, so make sure the participants are prepared not to inadvertently show things like:
- the applications they use
- the mobile device home screen’s background image
- browser information
- recent URLs, when completing a task that involves using search.
Provide plenty of verbal reassurance
When screen sharing, the participant can only see their screen and not the moderator’s, which can make them feel awkward and isolated. So, provide plenty of verbal encouragement — let them know you’re there and you can observe what they’re doing.
What we learned about the mobile experience
Testing on mobile was a real eye-opener. We’d made assumptions based on our own experience testing on desktop using mobile view. But of course, the mobile experience is significantly different.
When designing a page you should consider:
- information seeking behaviour and time spent on page — it takes around 50 milliseconds for people on mobile to decide whether a website meets their needs
- physical interaction — flick-scrolling increases the speed in which people can move around a page compared with using a mouse to scroll, or keyboard to tab
- how content is presented on mobile — content is compressed into a single column rather than spread out across a page, meaning less content is immediately visible.
How participants navigated
All participants navigated using the:
- breadcrumb menu
- headings — people skim read the page headings and subheadings to find what they were looking for. This validates good content design practice of structuring pages with specific meaningful headings.
Participants did not recognise the site menu when concealed in the accordion (disclosure widget) at the top of the page in mobile view.
Scroll behaviour
Most participants:
- did not scroll to the bottom of the page or footer of the page unless prompted and so missed the contact information
- rapidly flick-scrolled the page and navigated away quickly if they couldn't find what they wanted.
Conclusions
- Test more — the only way we can avoid assumptions is to test with people.
- Use the tools you have — you do not need much to get great insights.
- First impressions matter — people are quickly put off if they cannot find what they’re looking for. So put the most important information at the top of the page and use meaningful front-loaded headings.
Next steps
- Iterate and prototype on the design and location of contact information.
- Test to make sure draft designs and content are tested on different devices to see how they perform.
- Improve how we conduct usability testing and user research using different devices.
Get involved
If you’d like to take part in user research, we’d love to hear from you.
Email diaresearch@dia.govt.nz
