📹 New! Remote User Testing - Get video + voice feedback on designs and prototypes
Request Demo

What is Usability Testing?

Usability Testing 101: Uncovering problems and opportunities in design

Since the early 90s, the terms ‘Usability’ and ‘User Experience (UX)’ have been talked about non-stop in the world of design, and are often referred to as ‘must-haves’ in the design process for products and services on multiple platforms. Where this couldn’t be more true, it’s hard to overstate the importance of usability testing. In this article, we’ll cover the importance of user testing and its benefits for both the end-user and your business.

It is easy to mix the terms up, but usability is one of many factors that shape the UX of a product, along with user research, content strategy and visual design. Creating an intuitive application or website takes time, and what may seem obvious to one person may not be for everyone.

 “Good UX design is based on how seamlessly a user can navigate through a site or use a product correctly.”

What is usability testing?

Usability testing is one of the most common techniques to check how useable your app or website is. This testing involves users from your target audience to accomplish a set of tasks in a controlled scenario – like signing up or making a purchase. Observers will take notes of where users have succeeded or where they have struggled, so they can revisit their designs at a later date and make improvements.

What usability testing is not

There are some UX tools that aim to improve customer experience but don’t qualify as ‘user testing tools’ because they don’t allow you to explicitly replicate the experience of real users when testing a website or mobile app for functionality. Here are a few methods that don’t qualify as user testing:

A/B Testing
Also called Split Testing, this method is to compare multiple versions of a web page to figure out the better performing version, but it won’t tell you why which is a major factor of why this method can’t be used for usability testing.

Heat maps
A heat map is a colour-coded overlay of how users move their mouse around the page by showing the hottest (most popular) and coolest (least popular) parts. These maps will show where people are aggregating on a website, but the accuracy of the data is questionable as many people don’t hover over web elements they are reading.

Surveys
Surveys may be used at the same time, but as they don’t allow you to observe users on the site in action, and for this reason are not considered a method of usability testing.

How is usability testing different from user testing?

Both user testing and usability testing are necessary for UX design, which creates and enhances the quality of the user’s experience in meaningful and valuable ways.

“User testing is the process of collecting qualitative and quantitative data from the test user, whilst the user is subject to all aspects of a service or product.”

User testing is the umbrella term for several types of methodologies that aim to help identify your target users, understanding their needs and behaviours, which includes:

“User experience (UX) design is the process of creating products that provide meaningful and personally relevant experiences. This involves the careful design of both a product’s usability and the pleasure consumers will derive from using it.
– Interaction Design Foundation”

A great way to separate the two terms is:

User testing gives you a deeper understanding of users’ needs and behaviours.

Usability testing helps you find the functionality issues in your design that you never expected.

Why is usability testing important?

You might wonder, why bother? Usability testing takes time and when you’re under pressure with looming deadlines, you might be tempted to forego it. Don’t make this mistake – it will cost you more in the long run. Usability testing will require a degree of time and money investment, but it will pay off.

The goal of usability testing is to gather as much feedback as you can on how real users interact with your product. As early as possible, and as often as possible. This will help you identify any design issues before you get to the expensive part of the process when you’ve reached the final build. If you leave usability testing until after you’ve built your product, then any further changes will be costly.

“To run an effective usability test, you don’t need anything fancy. It’s far better to run with whatever you have to hand than to run no usability testing at all.”

What are the benefits of usability testing?

No matter where you are in your design process, from prototyping all the way to the finished product, your website or app can still benefit from usability testing. This offers a variety of benefits for both future users of a product and the company creating it, for example:

Types of usability testing

types of usability testing types

There are 3 types of usability testing types:

Remote vs In-person
As the name for both suggest, remote usability testing is done over the internet or by phone and in-person testing requires the test to be completed in the physical presence of the moderator/researcher.

Traditionally an in-person test was more popular because researchers could observe and analyse body language and facial expressions. However, with the advancement of technology remote testing has increased in popularity with the use of video. Some researchers may even say that remote testing is better, as it allows you to test a larger number of people in different geographical areas using fewer resources than in-person testing.

Moderated vs Unmoderated
Moderated testing is when the session is managed in person or remotely by a researcher who introduces the test to participants, answer any of their questions and asks for follow-up questions. Because of the direct interaction between researchers and participants, these tests are usually held in labs and produce in-depth results but can be expensive to organise and run.

On the other hand, unmoderated tests are done without direct supervision and it’s more likely that the tester will be in their own homes and or use their own devices to test the website or mobile application. The cost of this testing is much lower, but answers can be vague and follow-up questions are impossible.

As a general rule, moderated testing is used to investigate the reasoning behind user behaviour, and unmoderated testing is to test very specific questions or to observe and measure behaviour patterns.

Assessment vs. Comparative vs. Explorative
These three testing methods will generate different types of information:

Qualitative vs Quantitative

A common question that everyone asks is “Is usability testing qualitative or quantitative?” In truth, it can be both.

Qualitative usability testing focuses on collecting insights and findings of how people use the product or service. It is best used for discovering problems in the user experience, and this form of usability testing is more common than quantitative usability testing.

Quantitative usability testing focuses on collecting metrics that describe the user experience. This is best used for collecting benchmarks and the two most commonly used types of metrics collected are task success and time on tasks.

The number of participants needed will depend on the type of study, but for a typical qualitative usability study of a single user group, the recommended number of participants is 5 and that will help uncover most of the most common problems in the product. For a quantitative study, the minimum number of participants could range from 5 to 50 depending on the scale, but most researchers would regard 100 participants as the minimum sample size when the population is large.

“Testing with 5 users generally unveils 85% of usability problems.” – NNGroup

Usability testing for mobile applications

Usability testing for mobile applications

In a world dominated by smartphones, engaging with mobile customers is becoming increasingly important. Whilst an app’s user interface is crucial, usability should always take precedence, as it’s every bit as much about good design as it is about pure functionality. Ignoring this during the design process will put your app at risk of deletion or worse, bad reviews.

A mobile app’s success hinges on one principal thing: how users perceive it. Usability contributes directly to how a user feels about your app, as they consider ease of use, perception of value, utility, and the efficiency of the overall experience. Usability is what will help convert users into loyal, long-term customers, generating more revenue for your app.

Test, test and test again

Usability testing is your chance to understand how your users perceive and use your products. By regularly inviting users to test your ideas, you’ll be able to meet their needs with actionable feedback, whilst validating your own designs, saving time and creating fewer headaches for the development and design teams.

“Regardless of the platform you are creating for and the industry that you are in, good usability requires thorough user research and an iterative approach of constant testing and refining.”

Tips to help you improve your user testing

Testing isn’t something that you can avoid, and even a simple round of testing could make or break your product idea. Here are a few quick tips to help you keep in mind when thinking about user testing:

Next steps

The User Testing Field Guide - Marvelapp

If you are ready to run your own usability testing sessions but would like more information on which type of usability testing method to choose, then we’ve got you covered with our free ebook, The User Testing Field Guide. It is for individuals and teams who have always wanted to get started with usability testing but didn’t know where to start.

And for those who say that they have no time, then check out our 8-step guide on how to do user experience testing, in our User Experience Testing Checklist, suitable for those fast-paced designers!

Further reading:

Content and Social Media Manager at Marvel | Photographer | Sewer

Related Posts

Guerrilla warfare: an independent group of paramilitary fighters using unconventional tactics to divide and conquer an often larger, more orthodox enemy. Guerrilla marketing: a group of extremely savvy execs using unconventional tactics to divide you from your cash and conquer your custom. (Think: the zebra crossing painted to resemble a carton of McDonald’s fries or the part unwrapped KitKat bench.)… Read More →

What are the benefits of automation and the steps to take towards an automated user testing process?

When it comes to attracting, delighting and retaining customers, user experience design is often at the heart of your success. Improving the usability of your app, website or product, means your end-user will be able to navigate and achieve exactly what you intended them to. This is often perfected through user testing, a method long proven to test and improve… Read More →