

perception
	as inspiration
	as evaluation (metrics and *methodology (new)*)

visual design
	as inspiration
	as evaluation (our new stuff)

evaluation alone!



10:42 - 11:00 (18 minutes on 2nd pass)
11:00 - 11:15

9:38 - 9:48 (break) 10:15 - 10:23  10+8=18 (yikes...)



\oo [title] overview
\ooo intro self
\ooo history of 3 topics at vis: visual design, evaluation, and
perception, at vis and infovis
\ooo some of our recent work at brown to relate these  areas

\oo [tiny text]
\ooo the history part
\ooo panels, keynotes, capstones from vis and infovis
\ooo slide cries out for a visualization

\oo [first graph]
\ooo summarizes 3 areas (vert axis) over the 16 year history of the vis
conference (horiz axis)
\oooo visual design knowledge applied to scientific visualization
\oooo human perception or perceptual psychology knowledge applied to
scivis
\oooo experimental evaluation used to test scientific visualizations
\ooo characterize the dangerous assumptions
\oooo vis conf (w/infovis) is all that matters
\oooo panels and invited speakers capture some of the themes of the time
\oooo the written record is sufficient (because no one changes what
they are going to say in the months between writing and speaking, right?)

\oo a few examples of specifics that illustrate some overall trends
\ooo [2nd graph] perception and vis design were mentioned right from the start
\ooo [3rd graph] real details about the human perceptual system came in a few
years later, and continued on every 1-3 years
\ooo [4th graph] evaluation and design combined were mentioned the year later --
occasional combinations every 6 years
\ooo [5th graph] then the first detailed look at experimental evaluation, both
from the perspective of anecdotal scientific users (4 on panel) and
from the perspective of quantitative human subjects experiments.  Al
Globus said ... .  This has been a sparser theme, only every 3-4 years
\ooo [6th graph] in '98 Pat Hanrahan illustrated the theme of visual design and
inspiration from art and art history.  This theme has continued almost
yearly
\ooo [7th graph] an interesting variant of this art and design theme has been
talks like Ben Shedd's, where the connection to scientific
visualization was left mostly as an exercise to the audience.
\ooo [8th graph] analagously, just last year, Bill Geisler
presented some of his human perception research, again leaving much of
the application up to the audience.

\oo [url] all writeups on the graph at: http://vis.cs.brown.edu/resources/lists/art.html

\oo [eval/percep/design circle]
\ooo they really all relate to one another
\ooo illustrate one connection among these three areas with work that
we have been doing at Brown
\ooo experimental evaluation and visual design
\ooo the one connecting top to bottom in my earlier graphs, so least
easy to see
\ooo four topics

\oo [eileen in cave] first, hiring visual designers to participate in
the visualization process
\ooo they know perception in a way that perceptual psychologists don't
\ooo they had some inspirational ideas, like using Miro as a model for
visualizing fluid flow around a bat
\ooo in general, though, their ideas couldn't be implemented fast enough
\oooo they couldn't get close enough to implementing the real thing themselves
\oooo having someone else code it up was too slow
\oooo the designers ended up idle, frustrated, and feeling
disconnected
\oooo takehome: we need tools to really enfranchise visual designers
in the visualization process

\oo [people in cs237 shot]
\ooo second brought into a visual-designers-with-cs-people class, co-taught
\ooo the 2nd time the class was offered, we built some tools to
enfranchise the design students
\ooo [cavepainting] built on CavePainting
\ooo couple real data into the painting process
\ooo [bat pics and anims?] show some bat pictures, and maybe an
animation
\ooo still just a start, designers still, at the end, were not first
class citizens

\oo [2D vector vis evaluation w/designers]
\ooo third, we had visual designers subjectively estimate the
measurements we made during
a user study we had run earlier
\ooo their evaluations matched closely what the much more cumbersome and
time-consuming user study had shown
\ooo better yet, unlike quantitative user studies, the designers could
not only match the results, they could often say *why* we got them

\oo [teach the computer -- picture from poster]
\ooo fourth, building on that, we are having visual design teachers
*teach* the computer what works
\ooo why teachers?  they can evaluate and verbalize what works and
doesn't work
\ooo they ``teach'' via critiques of what the computer can generate
(just like they teach students)
\ooo they are evaluating visual elements and combinations of elements
-- by capturing this knowledge, we hope to be able to build better
visualization methods
\ooo plug Daniel Acevedo's poster

\oo [takehomes]
\ooo better enfranchise visual designers
\ooo have them in the evaluation process
\ooo have them design
\ooo create better tools for them to use in the design process
\ooo train them to communicate with scientists (and vice versa)




Where Have We Been and Where Are We Going?

Over the last six years keynote, capstone, and panel speakers at
Visualization and InfoVis have opined about various aspects of visual
design and evaluation. Perceptual psychology, which links the two
areas, has also been a recurring topic. I will give a historical
perspective of these sessions, recognizing that they represent only
the recent past, not the origin of these ideas. Some timely themes
that have emerged include the value of engaging visual designers in
the visualization development process; the utility, complexity, and
broad gamut of evaluation methodologies; and the connections among all
three areas: design, evaluation, and perception. I will explore some
of these themes with examples from our work at Brown. In particular, I
will describe several projects where we, as computer scientists, have
been collaborating with visual designers.  In one project, we have
co-taught a ``Virtual Reality Design for Science'' class.  The class
comprised half Brown CS students and Rhode Island School of Design
(RISD) students.  Tools for design have been the primary limitation in
making such collaborative efforts successful.  In another project we
are modeling the expertise of visual designers from RISD.  Early
results are promising for automating the generation of effective
visualization designs.


================

2003
Which Comes First, Utility or Usability?
Chair: Georges Grinstein (University of Massachusetts)

1998 Capstone Address : Visual Explanations, Edward Tufte,
Yale University

1997 Capstone Address: Drawing on the Right Side of The Brain
	Betty Edwards, CA State University

1995 Panel: Metrics and Benchmarks for Visualization (Abstract)

Chair: Holly Rushmeier
Panelists: Mike Botts, Sam Uselton, Jeremy Walton, Dave Watson, and Howard Watkins
