User-Centered Evaluation of Visual Analytics

Jean Scholtz is a chief scientist at the Pacific Northwest National Laboratory (PNNL). Dr. Scholtz came to PNNL in 2006 to work on the Visual Analytics Science and Technology (VAST) Challenge for Jim Thomas, as part of the National Visualization and Analytics Center (NVAC). Although she no longer works on the VAST Challenge, she continues to advocate and conduct user-centered evaluations for various programs at the laboratory. Dr. Scholtz previously worked at the National Institute of Standards and Technology (NIST), where she worked in user-centered evaluations for an Intelligence Advanced Research Projects Agency (IARPA) program, Novel Intelligence from Massive Data (NIMD), and collaborated with PNNL to start up the VAST Challenge. At NIST she was a founder of the Common Industry Format (CIF) project that established a format for software companies to describe their usability testing results. This document is used in requests for information today by companies interested in purchasing software and has become an ISO standard. Dr. Scholtz also worked on human-robot interaction evaluations while at NIST, helping with both Urban Search and Rescue (USAR) and Explosive Ordinance Disposal (EOD) testbeds. She was a program manager at the Defense Advanced Research Projects Agency (DAPRA) involved in collaboration software, digital libraries, and ubiquitous computing. In an earlier career, Dr. Scholtz worked at Bell Telephone Laboratory in Murray Hill, NJ, where she helped develop an early time-sharing system, Project Mac, with researchers from MIT. She also worked on early efforts at missile defense on Kwajalein, MI. Dr. Scholtz has a Ph.D. in computer science from the University of Nebraska at Lincoln, a Master's degree in mathematics from the Steven Institute of Technology in Hoboken, NJ, and a Bachelor's degree in mathematics from the University of Iowa. Dr. Scholtz has been a member of ACM since 1988, has served on a number of CHI conference committees, and was a member of the SIGCHI Board. She received the CHI lifetime service award in 2014. She served for a number of years on the NASA Human Factors in Space Review Board.

• Acknowledgments.- Introduction.- Analysis.- Analytic Methods.- What is Visual Analytics and Why is it Needed.- User-Centered Evaluation.- Evaluation Needs for Visual Analytics.- Current Examples of Evaluation of Visual Analytics Systems.- Trends in Visual Analytics Research and Development.- Conclusions.- References.- Author Biography.