The principle of usability testing: a logical trade-off

Article Description: the trade-off between usability testing.

For usability testing, there are some universally accepted principles in the industry. They are as sacred as the theories of the natural sciences, and it seems that we can only practice the "good usability test" by doing what we do. In fact, even science, one of its characteristics is also "falsification"-the correctness of the theory is always a prerequisite. The truth is one step further and it becomes a fallacy!

The same is true in usability testing, which requires flexibility in terms of purpose, resources, and environment, rather than adherence to one or a few principles, perhaps as a manifestation of the importance of usability practitioners ' experience.

A Task settings: Fine VS Broad

The task is too elaborate and is generally opposed in principle. The reason is clear, if your task fine to step-by-step "guide" user to operate, that is too inconsistent with the user's reality in the use of the situation, usually no one next to "guide" the user's every step of the operation, but also too much control of the user's operating procedures, users lack the flexibility of real use.

is not the task we set can only be broad, can not be refined it? This must be done according to the purpose of the study. If the product is in the initial stage of design, we need to pay attention to some grand problems (such as: the overall structure of the site, navigation and classification of the rationality, the logical relationship of the page), at this time need to pass a broad and flexible task to find the macro-level issues. If the design of the product is already perfect, begin to modify the details of the iteration, at this point, you need to set a relatively specific task to find specific details (such as: the understanding of a name, the use of buttons, link clicks, Form Fill). According to Don T makes me think: The average user is satisfied with the use of the Internet products, will not seek the best way to use; only scan the Web page, do not read carefully. So, if you set up a task in a completely broad and flexible way, although it's more consistent with actual usage, it's likely that users will skip over the details you want to investigate.

In actual work, due to time and resource constraints, can not do each product from the initial design to the on-line before and after a number of usability testing. You may need to focus on both macro and detail issues in one usability test. At this point, it is necessary to communicate with product managers and interaction designers repeatedly to confirm the main purpose of the test, and to make the secondary objective as satisfying as possible by weighing the precise level of the task.

However, even if you want to investigate the details of the task, but also try to avoid the "direct operation" style of language description, so that the task and real use of the situation will not be too far apart. For example: Want to investigate the Watercress reading page "Want" button can be seen, whether have a clickable sense. Here are two ways to make comparisons:

A. Please find the book you like and click "Want" on the page. (x)

B. Please find the book that you like and mark it on the page. √

Two Number of tasks: more vs Less

The number of tasks is related to the scope of usability testing, and to the precise degree of the task. The number of tasks required is naturally different if the site is inspected and only one of the pages and one of the procedures are inspected. Under the same scope of study, if the task is set more finely, the number of tasks required will be more.

Studies in Lindgaard and Chattratichart (2007) found significant correlations between the number of tasks and the percentage of usability problems found (r=0.82,p<0.01). In order to find as many usability problems as possible, do we try to set up the task to the user as many?

Consider the drawbacks of the excessive number of tasks: learning effects and fatigue effects, especially when tasks are more likely to be affected. The way to deal with this problem in psychological experiments is to balance the order and counteract the effect. However, the scenarios and tasks set up in the usability test are in a particular order and are not suitable for sequential balancing methods. Based on our experience, or through the test of the number of tasks to control, to ensure that the formal testing links up to 1 hours, plus the front and back of the welcome language, interviews, questions and answers, the entire process is not more than 1.5 hours.

In addition, the number of tasks will indirectly affect the number of participants required to test.

Three Number of users: 5 enough vs 5 Not enough

Nielsen's study found that 5 users could find more than 80% usability problems. This conclusion is respected by many people, so called "magic number 5". The source of this conclusion is that each user can find 30% usability problems on average, and assume that all problems have the same probability of being found. However, this premise may not be valid when the number of tasks set is too high and the task is of varying sophistication and difficulty.

Studies in Lindgaard and Chattratichart (2007) found no significant correlation between the number of tested users and the percentage of usability problems found. This conclusion seems to support our choice of a small number of users to test it.

In fact, the user recruitment phase, more than the number of users need to pay attention to the problem of user representation. The ability to recruit a representative user will directly affect the success or failure of usability testing. If testing a medical software product, recruiting health care personnel and patients as test users, the 5 users may be enough, but if only a medical intern is recruited to test, there must be more than 5 users (even if this does not necessarily infer the entire product's user base).

In view of this, the number of users recruited and the number of tasks, the level of precision, the user's representativeness is also closely related. Reference Tom Tullis (2009) and my experience: when the usability test scope is limited to a certain range (20 tasks, or 30 pages), and recruit a strong representative of the user, then 5 is enough. If there are different subgroups, try to achieve 5 or so representative users per subgroup (of course, the characteristics and classification of the target users should be the problem before the usability test in the user research phase); No more than 12 users in one test.

Four User performance: Behavior vs. speech

There is no doubt in the usability test that the focus on user action behavior is emphasized. Because:

1. The user's behavioral indicators are more clear, specific, objective, easy to observe and record.

2. If the focus is entirely on the user's operational behavior, then there is no need to communicate with the user unnecessarily (outside the lead) language. Similar to the norms of psychological research, the instruction in experiments or tests is unified, and all unrelated variables (including the language and posture of the main test) are controlled to reduce interference in the research process.

3. Even if you ask the user some questions directly, you are most likely to get the wrong answer. The experiments of Richard Nisbett and Timothy Wilson 30 years ago, and the writings of Peter Johansson 2 years ago in science, confirm that in some cases people cannot explain the real reason for their actions. In addition, users may try to figure out the preferences of the main test and answer what they think the main test expects.

Therefore, it is necessary to emphasize that the focus of the usability testing process should always be the user's behavior and minimize interference from any extraneous variables. But this principle has been extended to the extreme by some people, that only observe the user's action is meaningful, other information is no need to pay attention to, and even rashly suspected that the user's words are not credible.

The primary purpose of usability testing, though, is to identify problems, but we also need to understand the reasons behind the problem, but only by observing the user's behavior is not aware of all the reasons behind the problem, at this point, we hope that users can use the "sound thinking Method", voice thinking is focused on how to interact with the product stream of consciousness. If the atmosphere in the test is more equal, natural, and harmonious, and the user is particularly willing to express, then the user will be in the task of the operation at the same time, to express what they want to do, how to do, behind the reasons. At this time, not only is the operation behavior, the user expresses the thought and the reason, as well as in the language reveals the doubt, the disappointment, the dissatisfaction, the surprise, hesitates and so on emotion also needs our attention. However, some users are more introverted, not good at the initiative to express their ideas, at this time need the main test and his simple communication to guide users to say the reason behind (note: Not to guide users to say you expect the answer).

Therefore, in the actual usability testing, the basic should focus on the user's behavior, a small amount of timely inquiry and communication is needed. But how can this degree be grasped?

1. When the user appears hesitant, surprised, task failure (the process node appears naturally slightly interrupted/paused) when the simple query.

2. Ask the sentence of the general interrogative sentence, repeat the behavior of the user just now (to be specific objective): "You just did not ..., right?" "-Although not directly asked" why ", but hinted that hope to hear his further explanation.

3. If the user does not own the initiative to say the reason, you can "by the way" ask why? "or by nonverbal cues such as body tilt, eye gaze, and so on, you want to hear more." If the user quickly and firmly say the reason, then this reason is higher credibility, if the user hesitate, or difficult to say why, do not continue to cross-examine.

In addition to the above language, mood, behavior need to get attention, there is a special situation is to understand the user "not speak" language. For example, we anticipate that there is a logical irrationality in the classification of a two-level navigation tag and a first-level navigation label on the site, but the user is in the test and the navigation-related steps go smoothly and the user says nothing. This usually means that the user takes it for granted and does not affect the operation-you need to understand the user's "not speaking" language at this point. If you interrupt the user rudely and ask, "What do you think of the two navigation tags?" , it becomes a kind of inducement to ask questions.

To summarize the practical application of this part of the content:

1. The user's operational behavior is always the focus of usability testing.

2. Encourage users to adopt the "sound thinking Method".

3. Timely and small number of questions to the user, prohibit the same question repeatedly asked "why."

4. The use of truly "listening" technology to maintain and the user's communication status, rather than through too much discourse.

5. To observe and listen to the user's "not spoken" language in an open, not predetermined position.

When considering the principles to be followed in usability testing, it is necessary to understand the applicable conditions and the interaction between them and other principles, and combine the purpose, resources and environment of this user study to form an optimal scheme as far as possible. Because of the length of the blog, this is summed up so much, in the next article will continue to summarize the other aspects of the principles.

[1] [2] [3] Next page