Why You Need to Test Your Web Site with Real Users

by Lois Wakeman ([email protected])

Why You Need to Test Your Web Site with Real Users

I recently attended the Nielson Norman Group User Experience World Tour (see my review if you want to know more). I learned much about making Web sites more usable, but the most common theme was the importance of usability testing. If you think usability isn't important, a recent test of shopping sites showed that almost half the tasks set to the test shoppers weren't completed successfully – which equates to a lot of frustrated consumers and lost sales when scaled up to the global level!

Assessing the usability of your site

To assess how usable your Web site is, you can choose a variety of techniques. I hope to convince you that user testing is one of the most important. Below are some of ways you can assess your sites usability.

• User surveys and focus groups are marketing tools, to find out what a selected group of users remember about a Web site (rather than how they actually felt while they were using it). Naturally, the "feel" of a Web site is an important element of branding and corporate image. A good feeling can increase trust in the site, so we should not dismiss it.
• User feedback solicits the views of a self-selected user group. Caroline Jarrett (https://www.effortmark.co.uk/) says that experience shows that, usually:
  • Visitors who hate the site leave promptly without responding,
  • Visitors with minor problems: broken links, typos etc. Often provide feedback, as do those who love the site,
  • The majority probably think the site is OK and don't bother to tell you this. Therefore feedback isn't a reflection of the overall spectrum of user experience, though it can highlight specific problems.
• Webmasters may rely on mechanical aids such as log files and automated testing services (like www.usablenet.com). Log files tell you which pages are most visited, longest studied, first/last visited etc., but not whether users found what they wanted, or why they left. Automated services advise on predictable problems: broken links, and missing ALT tags for images, but are inevitably unable to spot complex things like difficulty in locating products, meaningless graphics, or poor results from search pages.
• Heuristic inspections are conducted by a usability expert or a team, systematically examining a representative selection of pages. Most people use checklists to evaluate against, though experience and instinct are equally important. The inspection report describes, at least, the most important flaws in the site, and may recommend improvements. However, they can cause problems - the creative and technical people in the Web team often resent the intervention of a know-it-all consultant! For more details of these and other methods, try the Information and Design or Usability 1st sites. Thus, marketing people, webmasters and consultants can help with usability, but with drawbacks. Let's look at that word again, "usability." The start of the word is the most important – the bit that's to do with the users. It is the user's experience of trying to do something with a site that is the critical thing to get right, and this can only be done effectively by testing with real users.

How user testing works

A small number of users (five are usually enough – see Jakob Neilsen's Alertbox) are asked, one by one, to perform set tasks using the site. Their behavior is observed (not directed!) by a knowledgeable person sitting in the same room, who takes notes on what they do and say. Sometimes, a video link or two-way mirror allows other staff to watch without disturbing the test.

The users are reasonably representative of the target audience for the site. If there are different target audiences, each must be tested separately. Testing can take place at almost any stage of development – in general, the earlier the better. Simple tests of early prototypes allow problems to be fixed, and then the next iteration of the site can be tested. This is a much more efficient use of resources than completing the site and doing one big test at the end.

Summary: Why you need to test with real users

Consultants, marketing and Web development people have their own biased views of what users want or need. But only users can really tell you whether your site meets their requirements.

However experienced the usability consultant who advised you on your site, there will always be some aspects that real users will find problematic. Caroline Jarrett, who's conducted many tests, says "In a test, at least one user will surprise you by using the site in an unanticipated way." Testing also has a useful side- effect: it can act as a neutral agent to convince your boss, your designers, the marketing team, and the developers of the importance of good usability. While anyone can legitimately question the opinion of the professionals, it takes real nerve to suggest that the intended users of the site don't know what they need!

This article originally appeared in the Jan. 11, 2000 edition of the WebReference Update Newsletter.

Comments are welcome

Revised: Jan. 17, 2001

URL: https://www.webreference.com/home/web/authoring/design/usability/testing/