Combining Competitive Analysis UX with User Testing for Better Results

Your team just finished a competitor feature matrix, and now someone asks the obvious question. Does it match what your users actually experience? A competitive analysis UX exercise tells you what competitors built, not whether your own product wins the same task. Pairing it with UXArmy’s usability testing platform closes that gap.

Competitive analysis UX combines two research tracks. One is a review of what competitor products do well or poorly. The other is usability testing that checks if the same tasks work for real users on your own product. Together, they turn a feature list into evidence you can act on.

Key Takeaways

• Competitive analysis UX shows what competitors built. A usability test on even five real users shows if your product handles the same task as well.
• UXArmy splits this into two methods. Comparative usability testing compares your own design options. Competitive usability evaluation compares your product against a named competitor.
• Gojek tested its live app next to competitor apps in Indonesia, Thailand, Vietnam, and Singapore in one research program. Its revenue grew 42.6% between 2021 and 2023.
• Run UX competitive analysis first when you already have an assumption about a competitor to check. Run a fast tool like UXArmy’s first-click test first when you don’t know what’s worth comparing yet.
• One shared task list and one UXArmy participant panel let you run both research tracks in a single round.

How Do Competitive Analysis and User Testing Work Together?

Competitive analysis and user testing answer two different halves of the same question. One shows what exists in the market, and the other shows if your own product handles the same task just as well, confirmed through a UXArmy test. UXArmy calls these two competitive usability evaluation and comparative usability testing, and its own blog keeps the two methods clearly separate.

Picture a product team that spent two weeks logging every checkout step across four competitor apps. The matrix is thorough. But nobody has watched a real user try to check out on their own app yet, so the matrix is still just a guess.

Here’s how competitive research UX teams split the two methods:

Nielsen Norman Group recommends testing against two to four competitors at most (NN/g, 2024).

When Should You Run Competitive Analysis Before User Testing?

Run competitive analysis in UX design first when you have a specific claim to check about a named competitor. Run user testing first, using a quick tool like UXArmy’s first-click test, when you don’t have that claim yet. Competitive analysis might mean checking if a competitor’s shorter checkout flow actually converts better. User testing means finding out what’s worth comparing before you spend the budget.

A fintech team once assumed a competitor’s onboarding won because it had fewer screens. A five-minute unmoderated test on their own onboarding found the real problem: one unlabeled field, not the number of screens.

• Run competitive analysis first if you have budget for only one round and a named hypothesis to confirm.
• Run competitive analysis first if leadership wants a market-position answer against a named competitor.
• Run user testing first if your own product’s baseline usability is still unknown.
• Run user testing first if the goal is fixing your own flow, not benchmarking the market.

Usability testing remains one of the top UX research methods. It’s used by 69% of UX professionals, according to UXPA’s 2024 salary survey.

How Do You Combine Both Methods in One Research Study?

You combine both methods with UXArmy’s platform by building one shared task list for your product and a named competitor’s app. Then you recruit a single participant panel and run every task against both apps in the same research round, not two separate ones. That’s the same setup UXArmy used for Gojek across four countries.

A researcher building a study like this starts with three or four tasks that matter most, like signup or search. Participants complete the same tasks on your product and the competitor’s, in random order, so the results don’t favor whichever app they saw first.

• Pick 3 to 4 shared tasks that exist in both your product and the competitor’s.
• Recruit one panel and randomize which app each person sees first.
• Run the study through UXArmy’s unmoderated testing on a live competitor app next to your own.
• Compare completion time, error rate, and first-click accuracy side by side, not as two separate reports.

The Gojek engagement used that exact structure. It included 46 usability tests and 28 user interviews across four countries in one research cycle. That’s what user testing and competitive analysis look like combined into a single plan, not two separate reports.

UXArmy Turns Competitive Testing Into Measurable Growth

Gojek needed to redesign its ride-hailing and food delivery app across four countries without delaying any of the launches. UXArmy ran 46 usability tests and 28 user interviews on the live Gojek app next to competitor apps in each market. It recruited local participants instead of guessing from one region. The combined research program fed straight into the redesign, and Gojek’s revenue grew 42.6% between 2021 and 2023.

Combining the Two Methods Is the Real Advantage

The real gain from combining competitive analysis UX, and user testing isn’t a bigger report. It’s fewer wrong bets on what to fix first. UXArmy’s own Gojek engagement shows why: a competitor matrix said what was possible, but only moderated interviews with real users showed what was actually broken for people using the app. So before your next competitor audit turns into a backlog item, which task would you test first?

Start Combining Both Methods With UXArmy

Run your next competitive round and your own usability test in one study, not two separate projects. See how it worked for Gojek and other teams in UXArmy’s case studies, then set up your own comparison.

Frequently Asked Questions