Usability testing: types, methods and when to use each
Usability testing comes in several types: moderated or unmoderated, remote or in person, qualitative or quantitative, and exploratory, assessment or comparative. Specialised forms include benchmark studies, A/B tests, guerrilla tests, accessibility testing and first-click tests. Teams choose based on the question, stage of the project, budget and how much detail or statistical confidence they need.
What usability testing is
Usability testing evaluates a product by watching representative users attempt realistic tasks. It shows whether people can complete what they set out to do, how efficiently they do it, where they struggle and how they feel about the experience. Unlike surveys or opinions, it focuses on observed behaviour, which makes it one of the most reliable research methods available. Our guide to running a usability test in seven steps covers the basic session process; this guide focuses on the different types of testing and when to use each.
Three ways to classify tests
| Dimension | Options | What it changes |
|---|---|---|
| Facilitation | Moderated or unmoderated | Depth of insight versus speed and scale |
| Location | In person or remote | Context and control versus reach and cost |
| Data | Qualitative or quantitative | Finding problems versus measuring performance |
Moderated and unmoderated testing
In moderated testing, a facilitator guides each session, reads tasks, observes and asks follow-up questions. This allows deep exploration of why people behave as they do and works well for complex products, early concepts and sensitive topics. In unmoderated testing, participants complete tasks on their own while software records their screen and answers. It is faster, cheaper per participant and easier to scale, but offers less opportunity to probe unexpected behaviour.
Remote and in-person testing
Remote testing uses video calls or online platforms, letting teams test with participants anywhere. It is convenient, often lower cost and lets people use their own devices in their own environment. In-person testing offers more control, makes it easier to observe body language and suits physical products, kiosks, older participants or situations where context matters. Many teams now run most tests remotely and reserve in-person sessions for specific needs.
Qualitative and quantitative testing
Qualitative tests, typically with five to eight participants per user group, aim to find usability problems and understand their causes. Quantitative tests use larger samples to measure performance with statistical confidence, such as task success rates, time on task or satisfaction scores. Qualitative testing is best for improving designs; quantitative testing is best for benchmarking, comparing versions and reporting to stakeholders. Our guide to measuring user experience explains common metrics.
Exploratory, assessment and comparative tests
- Exploratory: Early in a project, to understand needs and test concepts.
- Assessment: Mid-project, to check how well a design supports tasks.
- Comparative: To compare two or more designs or competitors.
- Validation: Before launch, to confirm the design meets agreed goals.
Specialised testing methods
| Method | What it involves | Best for |
|---|---|---|
| Benchmark study | Repeated tests with consistent tasks and metrics | Tracking progress over time |
| A/B test | Comparing two live versions with real traffic | Measuring impact of specific changes |
| Guerrilla test | Quick, informal tests with people nearby | Early, low-budget checks |
| First-click test | Checking where people click first for a task | Navigation and layout decisions |
| Five-second test | Showing a screen briefly and asking what people remember | First impressions and clarity |
| Accessibility testing | Testing with assistive technologies and disabled users | Inclusive design and compliance |
How often to test
Usability testing works best as a regular habit rather than a one-off event. Many teams run small qualitative tests every sprint or every few weeks on upcoming designs, benchmark studies once or twice a year on key journeys, and A/B tests continuously on live products where traffic allows. Regular testing catches problems early, builds a shared understanding of users across the team and shows how the experience improves over time.
How to choose a method
- Define the question: Are you exploring needs, finding problems or measuring performance?
- Consider the stage: Early concepts suit moderated exploratory tests; live products suit benchmarks and A/B tests.
- Check your users: Hard-to-reach or specialised users may need moderated sessions.
- Weigh time and budget: Unmoderated remote tests are fastest and cheapest per participant.
- Decide on confidence: Need statistics? Plan a larger quantitative study.
Using a testing service
Many organisations run some testing themselves and use external services for larger or more specialised studies. Testing services can recruit hard-to-reach participants, provide experienced moderators, offer objectivity, run studies in several languages and markets, and deliver analysis and recommendations. For Gulf audiences, look for providers that can recruit and moderate in Arabic and English. Our guide to UX research agencies in the UAE covers how to choose one.
- Participant recruitment and incentives
- Moderation in the right languages
- Lab or remote testing infrastructure
- Analysis, reporting and video highlights
- Workshops to turn findings into actions
Combining methods
The most reliable insight comes from combining methods. A team might run qualitative moderated tests to find problems, fix them, then run a quantitative benchmark to measure improvement and an A/B test to confirm impact on conversion. Pairing testing with analytics, as described in our guide to user research methods, shows both what is happening and why.
Frequently asked questions
What is the difference between moderated and unmoderated testing?
Moderated tests have a facilitator who guides and probes; unmoderated tests let participants complete tasks alone while software records them.
How many participants do usability tests need?
Qualitative tests usually need five to eight per user group; quantitative tests need larger samples for reliable statistics.
Is A/B testing a usability test?
It is related. A/B tests measure which live version performs better on a metric, but do not explain why, so they work best alongside qualitative testing.
When should we hire a testing service?
When you need hard-to-reach participants, multiple languages or markets, independent results or more capacity than your team has.
Can we test with internal staff instead of real users?
Staff can help with quick early checks, but they know too much about the product. Test with representative users for reliable findings.