Moderating Usability Tests

Joe Dumas is a recognized expert in usability evaluation. He has 25 years experience as a usability professional. He as moderated or observed others moderate thousands of usability testing sessions and taught numerous students and usability professionals how to moderate. He is the author of A practical guide to usability testing (with Ginny Redish), Designing user interfaces for software, and numerous articles, both for researchers and practitioners. He is currently a Usability Consultant for Oracle Corporation. He was a Senior Human Factors Specialist at Bentley College’s Design and Usability Center and taught graduate courses in the college’s Human Factors in Information Design Master’s Degree program. He has a Ph.D. in Cognitive Psychology.

"Joe and Beth really know their stuff, and they’ve put together a book that’s enormously valuable for usability professionals and usability amateurs. Whether you’ve conducted hundreds of tests or are about to try your first one, you owe it to yourself--and your team...and your test participants--to read this." --Steve Krug, author of Don’t Make Me Think"Interacting with participants in a calm and neutral manner may well be the most difficult part of doing usability testing. Now you no longer have to worry about how to do that. Just follow Dumas and Loring's wonderful, practical advice and you will be prepared not only for typical encounters, but also for the unusual and unexpected, for doing remote testing, and for working with special populations. Moderating Usability Tests is a great resource for anyone who interacts with usability test participants." --Janice (Ginny) Redish, President, Redish & Associates, Inc."Everyone talks about research methods, but the formal aspects of those methods only get you so far. The difference between getting a little data or a lot of data, only discovering problems or getting ideas about solutions, bias or validity, throw-away data versus generalizable insights, often depend on the soft skills, the ability to effectively moderate testing. In the past, you were expected to get these skills through apprenticeships or trial and error. Moderating Usability Tests: Principles for Interacting with Participants removes the mystery and provides practical advice on how to get the most out of research. It will be invaluable to students learning about usability testing for the first time, people newly charged with evaluating products, and even old hands looking to refine and improve their technique." --Arnold Lund, Director of User Experience, Microsoft"You may not think that being a “Gracious Host” is among your assignments in moderating a usability test, but you will learn why this and other roles with similarly illuminating names are important to your success. In this generous book, Dumas and Loring give the benefit of their decades of experience and astute observation of both the foundational and the subtle aspects of conducting usability tests. Many questions you didn't think to ask until you were on the hot seat are answered here, and will help you achieve a level of confidence as a test moderator that may have seemed beyond reach, even if your participants are from challenging-to-test populations. With this highly ethical and thoroughly grounded program for developing moderator skills and avoiding pitfalls, Dumas and Loring make a strong contribution to the body of knowledge on testing products. The big surprise of the book is that their clear, reasoned, and detailed suggestions about interacting with test participants and developers will likely spill over and improve your relationships with co-workers, family, neighbors, and friends." --Elisabeth Bayle, Bayle Collaborations"At this point, virtually everyone in the software industry knows what usability testing is. An unfortunate side effect of this awareness is that many people are conducting usability testing who have no idea how to do so in a way that will yield valid, reliable and useful data. Other than the design of the test itself, proper and effective moderation of test sessions is one of the most important - and least understood - aspects of usability testing. Here is a book by two highly regarded experts that covers this topic thoroughly in a very readable format. No one who has not already been well trained should attempt to conduct usability testing without first reading this book cover to cover, and viewing all the excellent videos the authors provide." --Deborah J. Mayhew, Deborah J. Mayhew & Associates

• Chapter 1: Introduction Introduction to the problemGoals of the book and videosA brief history of usability testing practicePart I: The rules of interactionChapter 2: Your roles as administratorThe gracious hostThe objective observerThe tradeoffs between themChapter 3: The Golden Rules - Part 1: the Five Core Rules1.The goals of the test and your relationship with developers determine why and when to interact2.Respect the participants? rights3.You have a responsibility to future users4.The participants are the experts; you are in charge5.Being ?real? is being professionalChapter 4: The Golden Rules - Part 2: Five Additional Rules6.Let the participants speak7.Your intuition can hurt and help you8.Be unbiased9.Don't give away information inadvertently10.Watch yourself to keep sharpPart II: Moderating the test sessionChapter 5: First contact ? Setting the tone Pre-test instructionsInformed consentChapter 6: Interacting During the SessionKeeping the participant talking How much to interactProviding encouragement Dealing with stressDeciding when to give assistanceHow to move the participant alongChapter 7: Post-task activities What to do firstTaking advantage of their knowledgePresenting ratings and questionnaires Making a good last impressionPart III: Special topicsChapter 8: Interacting in a remote testing sessionPreparing for the sessionGetting the session startedInteracting when you can't see the participantChapter 9: In the Room vs. Out of the RoomAdvantages and disadvantages of being in the roomAdvantages and disadvantages of being out of the roomChapter 10: Summing UpWhere do we go from here?Advice for novice administrators