ICE-T Value-Driven Visualization Evaluation

Team Members: Emily Wall, Meeshu Agnihotri, Laura Matzen, Kristin Divis, Michael Haass, Alex Endert, and John Stasko

Evaluation Methodology Details

    A Heuristic Approach to Value-Driven Evaluation of Visualizations - TVCG 2019
    Value-Driven Evaluation of Visualizations - BELIV 2014

Existing evaluations of data visualizations often employ a series of low-level, detailed questions to be answered or benchmark tasks to be performed. While that methodology can be helpful to determine a visualization's usability, such evaluations overlook the key benefits that visualization uniquely provides over other data analysis methods. We propose a value-driven evaluation of visualizations in which a person illustrates a system's value through four important capabilities: the time savings a visualization provides, the insights and insightful questions it spurs, the overall essence of the data it conveys, and the confidence about the data and its domain it inspires.

We have created a heuristic-based evaluation methodology to accompany the value equation for assessing interactive visualizations. We refer to the methodology colloquially as ICE-T, based on an anagram of the four value components. Our approach breaks the four components down into guidelines, each of which is made up of a small set of low-level heuristics. Evaluators who have knowledge of visualization design principles then assess the visualization with respect to the heuristics.