User Experience Research: the idea of online usability testing

I like the "Gossip Terminator" program because I think the challenge he faces is real. In the program, use a delightful and scientific method to test the message that people generally agree with. My favorite part of the show, of course, apart from the exploding part, is that what I believe is proving untrue. It keeps me open and focused on reality. I also like the other side-revealing rumors that have not yet been built up. This is the perspective of my article. In particular, I will highlight five common rumors about usability testing and explain why I think these rumors are untrue.

The motivation for writing this article is to help users to open their minds when they experience online usability testing by researchers. Many researchers have used this method for many years and are useful (in some cases). Others are just beginning to contact, and want to know more about its strengths and limitations of information. Finally, some user experience researchers have developed some ideas about online usability testing and think they are not useful for a variety of reasons (not formed). I want to expose these rumors through me, and as a user research group we can evaluate this tool based on real merit.

1, there are a large number of interference factors affecting the reliability of data

Perhaps most of the rumors about online usability testing are that the data are not trustworthy. Some people think that these participants will not work very hard, and will be distracted when they participate in the research, to complete the research as quickly as possible to get the appropriate incentives, or to falsely state the quality of their research (the aim is to get rewards). This is true, and in general the 5%-10%in all cases. But that's a good news.

Here are some good techniques to get rid of this data. Identifying (and removing) "mental cheats" is not difficult. The behavior of these people can be predicted. "Flattening" means that some people's answers are 1 or 5, in 5 rating scales. Speed detection is a way to detect these people (see Problem 6 in this picture) and make sure they really read every question carefully. Consistency testing can be used to analyze the consistency of the answer by repeating the word problem. It is also easy to remove unrealistic task times and to help deal with participants who come home from the middle of a task weekend. At the same time, screening questions can be written to the least deceptive participants in the research (those who disguise themselves).

In the final analysis, online usability testing data can be as reliable as traditional lab testing. The only difference is that you need to do some cleanup before you jump to the analysis stage.

2. You just need to collect data .

I was amazed when people told me that online usability testing only applies to collecting basic data: such as the success of a task, the time of completion, the degree of satisfaction, and so on. For some reason people think it's just because you have a large sample size , You can only collect quantitative data. Network usability research is meaningless If you want to collect qualitative data about the user experience . These cannot be far from the truth.

One of the benefits of online usability testing is that researchers can collect a lot of quantitative and qualitative data about the user experience. For example, if a user has difficulty with a particular task, they can provide annotations that explain why they have difficulty with these tasks, or what they want to do when they have difficulty with the transaction. Qualitative data, often detailed reviews, can be collected at any point in their experience. When users show certain behaviors, they may even be prompted to provide some detailed comments, such as abandoning the transaction halfway or straying from the navigation behavior.

Detailed comments are not only easy to collect, but are easy to analyze. There are a number of available ways to pick and trace these detailed comments. In a detailed answer, the phrase group (see below) is a way to get a critical model of rapid response.

3. Online usability testing does not achieve good results in the design phase

Most people think that running online usability tests only works when the product is established. Such online usability studies are only one part of the summative assessment and are not suitable for the actual design phase. Online usability testing does not guide the design, either spending too much time or spending too much at the design stage. These are not practical.

Online usability testing can be built within a few hours, and data can be collected quickly. We've been able to build, publish, and analyze data from online usability tests in one day. Not only can it be built quickly, but it can also help answer the questions that occur at the design stage. For example: Questions about the selection of parameters for navigation, labels, appearance, and feelings. There are other design parameter selection problems that cannot be answered in a small sample of two. Sometimes we need to quickly collect data from tens of thousands of users in order to identify major design scenarios.

4. It only applies with the website

The researchers believe that online usability testing only applies to site evaluations. It cannot be used in software, voice response systems, mobile phones, appliances and ovens. While a lot of online usability testing is focused on the site, it's not necessary.

Over the years, we have used online usability testing methods that are not limited to the evaluation of Web site interaction interfaces. Theoretically, this is still the same approach. Participants are asked questions, given tasks, and provide feedback on their real experiences. The only difference is that you can't track the behavior involved. However, you can collect a lot of data about their experiences, such as whether they can successfully complete their task, how long they have spent, how they feel about the experience, or what difficulties they have. So even if you may give up some behavioral data, there is still a lot of their experience to study.

5. online usability testing costs too much

Some people say they want to do a lot of online testing, but regular usage is expensive. It's really expensive, but luckily, there are a variety of tools available in the last few years that make online usability testing no longer a huge expense. There are a number of self service providers that allow you to build and conduct online usability research. You just have to pay them for the technology. Please visit www.measuringux.com to see the list of suppliers.

If the budget is really tight, you can use some discounted online available line test methods, or basically free. With some online measurement tools (such as investigating Gizmo or investigating monkeys), some HTML and JAVA scripts, you can build your own almost free online user research (see the figure below as an example, is the local method). And when you don't have a budget for online usability research, it's necessary to give up some data and functionality in this situation. Please visit www.measuringux.com and you can see more detailed instructions on how to create a discounted online usability test.

Give it a try.

I may not necessarily be able to convince you that these rumors are untrue. Anyway, I support you to use online usability testing as part of your user experience design and research effort. By doing so, you will find its advantages and limitations. After all, every user research approach has its own advantages and disadvantages, and online usability testing is no exception.

Editor's note: If interested in this subject,Bill has recently written "Beyond Usability Labs: directing a larger scale of online user experience research"

Translation: UPA editorial group-Mo Yan

Related connection: More:read all the five myths of online usability testing