A first foray into user testing
17 December 2004
It may shock, surprise and disappoint you, but I did formal user testing on a website for the first time ever last week. Sure, I'd gotten people to play around with a site and asked them the non-specific question "what do you think of this?" But this was the first time I'd been involved in rounding up a group of prospective users, sitting them down and watching their every twitch, mutter and groan as they jumped through the hoops we'd set for them. And it was fascinating.
Gratifying to see the users understand your carefully structured menus. Mystifying to see users type search commands into an enquiry form. Horrifying to see users wandering circuitously in search of a picture. As with anyone who's done testing, though, I'd highly recommend it.
There's been enough articles written by more experienced people than I on the subject of "guerilla" user testing:
- Builder.com: Guerilla Usability
- SitePoint: Professional Website Usability
- Webmonkey: Why User Testing Is Good
- WebReference: Why You Need to Test Your Web Site with Real Users
- Usbaility.gov: Conducting and Using Usability Tests
All those articles give you advice on taking usability out from behind the one way mirror, and making it easy (read: cheap) enough to be a viable development method on medium to large projects. However, there's a few things I'd reiterate from my own experience, and which I'll personally try to remedy in my own future dealings with user testing.
Firstly, it's impossible for a human observer to capture everything about what a user does – there's just too much going on to record every mouse movement, button press and search phrase; as well as timecode it all. If you want to capture such data, you'll definitely need to setup a video camera, otherwise, you'll just have to be satisfied with a record of what pages a user visited and what they clicked on. But if you want to find any major deficiencies in your web site, that's probably going to be enough.
Secondly, the wording of tasks can strongly influence the way that someone proceeds towards the goal. Depending upon what you want to achieve, you should try and eliminate any bias expressed in the words you use, and the way that you phrase them. For instance, if you supply a task loaded with keyword lists: "Find a news article about Cambodian locusts" then the tendency is to perform a search for those keywords. However, if you want the user to consider other navigation methods, or to formulate their own search phrase, then you should supply a less directed task: "Locate news about agriculture and how it is affected by crop pests."
There also seems to be two schools of thought to the observation process: get the user to verbalise what they are doing, and why, or let them traverse the site naturally. In this instance I opted for silence, reasoning that if a user has to explain their actions, those actions will be artifically influenced through some sort of Heisenberg usability principle. It was left up to an exit survey to glean the hidden meaning in a user's confusion. It seemed to produce some good results, but I can't be certain which method is better without seeing some user testing of user testing methods.
Oh, and you should clear the history cache in the browser, so that successive users don't get visited links leading them through your well organised tasks :o]
Follow me on Twitter
To hear smaller but more regular stuff from me, follow @themaninblue.
- Resolution dependent layout update
- footerStickAlt: A more robust method of positioning a footer
- widgEditor: A simple, standards-compliant WYSIWYG HTML editor
- Accessible, stylish form layout