Two papers for Tom Gay's talk on 2/14/96:<P>

<A HREF="#Color and Sound report">Color and Sound in Algorithm Animation</A> - 
Brown & Hershberger<P>

<A HREF="#Auralization report">Program Auralization: Sound Enhancements to the
Programming Environment</A> - DiGiano & Baecker<P>
<BR>
<HR>
<BR>

<H2><A NAME="Color and Sound report">Color and Sound in Algorithm Animation</A>
</H2>
        In this paper, Brown and Hershberger attempt to classify the ways in 
which color and sound, both independently and calaboratively, can be used to 
improve algorithm animations.  They review some of the general concepts involved
with algorithm animation and talk breifly about the effect of choice of input
data on an animation.  This sets a basis upon which the fundamentals of color
and sound are added, making ties to these underlying principles and conventions.
<P>

<H4>Techniques for Algorithm Animation:</H4>

<UL>
<LI>Multiple Views
<UL>
<LI>    Detailed view of a single algorithm
<LI>    Multiple algorithms simutaneously
</UL>
<LI>State Cues
<UL>
<LI>    Link views together
<LI>    Reflect dynamic behavior of the algorithm
</UL>
<LI>Static History
<UL>
<LI>    Allows user to process the algorithm at their own speed
<LI>    Allows the user to focus on key events
</UL>
<LI>Input Data
<UL>
<LI>    Small instances good for detailed understanding of low level operations
<LI>    Large instances good for finding patterns and general feel of algorithm
<LI>    Two types of useful input data:
<OL>
<LI>            Pathalogical data -> push algorithm to extremes
<LI>            "Cooked data" -> make the animation interesting
</OL>
</UL>
<LI>Continuous vs. Discrete Transitions
</UL>

        In classifying the many uses of color and sound in algorithm animations
these different aspects of algorithm animations in general will be utilized.  
The authors first present a number of challenges that the designer of an 
algorithm animation faces which he/she would not face if working on a paper 
design.  They then show how color and sound may be used to overcome some of 
these obstacles.
 
The problems that they sight are:
<OL>
<LI>Small screens - less real estate for presenting information
<LI>Lower resolution _ less detail can be packed in than on paper
<LI>Dynamic nature - the animation must be flexible
<LI>Multiple views - views must work together to provide a cohesive visualization
<LI>Multiple data sets - the animation must handle a range of data values
</OL>


<H4>Color Usage Techniques:</H4>

<UL>
<LI>Reveal the state of the algorithm
<UL>
<LI>    color provides an extra dimension - shape and color can vary
<LI>    a spacially denser presentation of information is made possible
<LI>    displaying global patterns is made easier
</UL>
<LI>Unite multiple views
<LI>Highlight areas of interest
<UL>
<LI>    provides temporary emphasis
<LI>    must maintain the integrity of the highlighted information
</UL>
<LI>Emphasize patterns
<LI>Capture history
<UL>
<LI>    color spectrum not intuitive to progression of time
<LI>    a gradation of hues can better represent history
</UL>
</UL>


<B>Question: The authors claim that the use of color allows a denser 
presentation of information.  Are there situations where this may be harder to
follow than the change of the shape or location of a graphical object?  How 
about for animations with a large number of these objects (<1000)?
</B><P>

<BR>

Before listing the ways in which sound can be used to enhance an animation, the 
authors first point out some of the limitations/considerations of using sound. 
<OL>
<LI>several users in a room or several animations on a display may produce a confusing com
bination of sounds
<LI>humans are not very well trained at discerning different sounds 
<LI>sound has an inherent temporal dependency
</OL>

<H4>Sound Usage Techniques:</H4>

<UL>
<LI>Reinforce visual views
<UL>
<LI>    associate sound with ordered value generated by important program event
</UL>
<LI>Convey patterns
<UL>
<LI>    This is done more efficiently by humans for large amounts of data
</UL>
<LI>Replace visual views
<UL>
<LI>    example - a tone associated with the number of active processors
</UL>
<LI>Signal exceptional conditions
<UL>
<LI>    everyday sounds are often useful here
</UL>
</UL>


<B>Question: What different types of sounds are best suited for representing
data? events? (ie. everyday sounds, beeps, speech, etc..) 
</B><P>

<BR>
<HR>
<BR>

<H2><A NAME="Auralization report">Program Auralization: Sound Enhancements to 
the Programming Environment</A>
</H2>

        This paper deals specifically with the use of sound.  It is not focused
on algorithm animations, however many of the techniques and concepts presented
could easily be used in designing them.  The authors first present how program
auralization might be encorporated into the programming environment, and then
present their system which is designed to do this.  Their system 
<I>LogoMedia</I> will not be described.

<H4>Definition:<P></H4>
Program Auralization - the use of non-speech audio for supporting the 
understanding and effective use of computer programs. 

<H4>Why use it?:<P></H4>

<UL>
<LI>the logarithmic nature of several sound dimensions can help teach logarithms
<LI>every-day sounds carry familiar conotations
<LI>smaller systems with smaller displays require other means of communication
<LI>cluttered graphics workspaces
<LI>sound can be varied across as many as 20 dimensions
<LI>sound does not depend on which way you are facing
</UL>

Each sound in program auralization will correspond to some item in a time 
varying stream of data.  Several example systems are given by the authors which
demonstrate some of the possible practicle applications already in use.

The authors present the notion of <I>articulatory directness</I>.  This is a 
measure of how well form follows function in an interface.  On one extreme is
a completely symbolic representation, while the other extreme is iconic.  
Program auralizations can be classified as representing data or events (or both)
some where along this access of articulatory directness.<P>

<H4>Sound in the programming domain:<P></H4>

<UL>
<LI>must be adaptable to varying conditions
<LI>must be able to represent data and events
<LI>must be able to handle unknown data types and sets
<LI>different programmers will want to use sounds differently
</UL>


The programming process can be broken up into three phases for the purpose of
program auralization: preparation, review, and execution.  The uses of sound
in each of these phases is outlined below.<P>

<H4>Execution:<P></H4>

Both values and events can be represented.  There are three classifications for
both values and events:<P>

<UL>
<LI>Common - typical data and control structures
<LI>Arbitrary - unpredictable elements of a program
<LI>Internal - the run-time stack, the heap, etc..
</UL>

Values:<P>

<UL>
<LI>frees the user to focus on other aspects of the running program
<LI>we can map values to many different sound dimensions
<LI>default sounds must fade into the background (must be unobtrusive)
</UL>

Events:<P>

<UL>
<LI>sound associated with a line or lines of code
<LI>recognition of patterns or "melodies" can be useful
</UL>

<H4>Review:<P></H4>

<UL>
<LI>analagous to indentation, comments, code style, etc..
<LI>audio landmarks can mark important code segments
<LI>by "scrolling" through code patterns can be recognized
</UL>

<H4>Preparation:<P></H4>

<UL>
<LI>layering of sound can provide syntax guidance
<LI>listening to compilation process can facilitate program understanding
</UL>

<B>Question: How might the use of sound in review of program source be useful?
What patterns does one expect to notice that would not be identified through
the use of good style (ie. indentation, comments, etc..)?
</B>