“You Need to Test with 5 participants enough” – the statement which so many people swear by! Why and how did 5 become the magic number for usability testing? The Common Industry Format (CIF, part of the U.S. Department of Commerce) for Usability Test Reports advises a minimum of eight participants! And, there are a few more widely accepted variations.
It is true that it takes 5 or 8 users (provided they are from your target audience) to test your application in order to identify a good majority of usability problems. However with the more commonly (mis)quoted 5-user rule, there are some caveats and it’s important for people who research, to clearly understand those.
Let’s revert back to the caveats in a bit. The question that the above discussion begs clarity on is “How many people to test with?”.
Short answer: It depends.
“Depends”? … on what?
That is one of the struggles UX Researchers have because there is no fixed and correct answer.
A number of factors come into play when deciding sample size. These could be one or more of the following questions that may help to decide the sample size for usability testing:
- What exactly is being evaluated? Design concepts, interactive prototypes, working products, etc.
- At what stage of the project is the usability evaluation being done?
- How many times the usability testing on the same product/material has already been performed?
- Is iterative usability testing on the same product/material already planned?
- What is at stake? How is the outcome supposed to be used? Initial Design refinement vs Product launch decision, etc.
- Who are the stakeholders involved? In the case of business decision-makers, the sample size must match the organization-wide (undocumented) magic number, below which the outcome might not be considered credible.
- What type of usability testing is being performed? Qualitative (focus on usage insights) or Quantitative (focus on usability metrics)?
- How complex is the study? Comparative / Benchmark studies need some quantitative results with statistical significance for decision-making.
- What is the budget and timeline?
When You Might Need More Participants
Quantitative Research: If your goal is to gather numerical data and get precise measurements of task completion rates or performance benchmarks, you’ll need a larger sample size, potentially 30 or more users.
Diverse User Segments: If you’re testing with users from different backgrounds or experience levels, you may need to test more participants to capture insights from each distinct segment.
Complex or Critical Products: For very complex interfaces or mission-critical applications, a larger sample size might be necessary to ensure broader coverage and uncover more subtle problems.
Problem Discoverability: The “Rule of Five” is most effective when problems are reasonably easy to discover. If a problem is very rare, you may need more users to increase the chances of encountering it during testing.
A factor that has been out of consideration in the above discussion is about how the usability study is being conducted. A selection between remote or in-person moderated usability testing may not affect the sample size significantly.
How many participants do you need for most usability tests?
However, selecting an unmoderated usability study calls for a slightly larger sample size. To understand this better, the unmoderated remote usability studies are performed without the supervision of a moderator. These studies make use of a usability testing software or user research platform on which the item to be evaluated is uploaded, users taking these studies at their convenient time. While the usability feedback gathered from this method has less bias due to a natural context of use, a few factors to be taken into account. Researchers may end up rejecting a small percentage of responses due to technological limitations, distracted participants or hard-to-comprehend voice feedback. Based on our experience with thousands of usability tests conducted on the UXArmy platform, we propose that the sample size for unmoderated remote usability testing must be 5 – 8% larger than the same study when conducted under supervision.
Back to demystifying the 5 user folklore, Jakob Nielsen is often quoted with his advice that “Five participants will discover over 85% of the problems”. Taking one step at a time, the NN group website clarifies that the 5-users statement was applicable to a specific context.
The Nielsen Norman Group website recommends applying statistical significance to calculate sample sizes while doing quantitative usability studies. For qualitative usability studies, there are a number of assumptions and a recommendation for doing iterative usability testing with 5 users. It is assumed that the nature of the usability study is qualitative and a number of follow-up usability testing iterations shall be made to thrash out the remaining usability problems in a Design.
In summary
The bottom line is, it takes more than 5 users in case your product usability is already good. It’s true that a larger sample size would be needed to find those hard-to-discover usability problems. So if usability testing is a practice at an organization and has been happening for the interface/product in question throughout the development, a larger sample size would be needed towards launch to ensure the remaining usability problems are found and if needed, fixed before the product release.
More participants don’t always mean that the research will be of a higher quality. Recruiting the correct participants in your usability study can meaningfully optimize your sample size without affecting the results.
That is why if you’re finding it difficult to recruit high-quality participants, we recommend tapping the help of research recruitment experts. By using a UX Research platform with a dedicated user panel, you can remove the hassle of finding the right participants by yourself.
UXArmy has a large pool of high-quality participants you can use for your usability testing needs. Visit uxarmy.com to learn more about UXArmy usability testing platform and our user panel known as the UserAdvocate community.
Experience the power of UXArmy
Join countless professionals in simplifying your user research process and delivering results that matter
Frequently asked questions
Is it okay to have 5 participants in qualitative research?
Traditional qualitative research practices often utilize fixed numerical thresholds—commonly suggesting sample sizes ranging from 5 to 30 participants per group [8].
What is a good sample size for usability testing?
Five Users
The Rule of Five: Why Five Users are Usually Enough. To understand why five users is the standard for most usability tests, it helps to look at a few simple numbers. Here’s the deal: Testing with five users will help you uncover an average of 85 percent of a product’s usability issues.
Is 5 participants enough for phenomenological research?
Similarly, five to twenty-five participants are suggested as adequate for phenomenological studies by Creswell (2013). Although small sample sizes are common in phenomenological studies, it is essential to emphasise that the researcher may need to keep adding individuals until saturation to fulfil the study’s goal.
What is the 5 user rule?
The 5-user rule is a principle proposed by usability expert Jakob Nielsen, suggesting that testing with five users is sufficient to uncover 85% of usability problems in a product. The reasoning behind this rule lies in the diminishing returns of usability testing.
How many participants for an UX interview?
(Study sizes of 6-20 participants were most common). Our sample size recommendation is based on my own analysis of 37 co-design studies involving 3-25 participants. Among these examples, the median number of participants per co-design study was 13, with 10 being the most common study size.
Are 5 interviews enough for qualitative research?
It’s often a good idea (for qualitative research methods like interviews and usability tests) to start with 5 participants and then scale up by a further 5 based on how complicated the subject matter is.
What is usability testing?
Usability testing evaluates a product’s ease of use by having real users attempt to complete specific tasks while observers watch their behavior and gather feedback. The primary goal is to identify points of confusion or friction in the user experience, which helps designers and developers uncover issues, improve navigation, and create more user-friendly and intuitive products before their official release.
