Discussion questions from CS 7390 lecture

Date: 2/14/96

Presented by: Tom Gay

Topic: Use of Sound and Audio

• It is hard to quantify sounds, but as explained in the paper, pitch or loudness are appropriate to represent logarithmic data. Is there "visual" data and "aural" data? How to quantify?
• Sounds can be used without moving our attention and gaze towards it. Is this really true? If you have lots of sounds will the center of attention shift from visual to aural input?
• The paper mentions about the difficulty of using a syntax-directed editor. Since I use one everyday, I don't see any problems. Can somebody shed more light on the above statement?
• I guess I found both these papers difficult to understand since I never heard any program auralization. Isn't it very subjective?
• In Mark Brown's paper, he gives an example of using tone to indicate the current number of threads running. When viewing the same information in a graphical display, it is usually easy to get quantitative information from the display through labels, lines indicating thresholds, etc. How could such quantitative data be "displayed" in an audio mapping?
• When I first heard the audio output from Brown's video on sorting, I thought it was pretty neat. The next few times I began to find it quite annoying and distracting. In had similar feelings about the video we watched with the hashing example, especially the car crash noises. Can there be a balance struck between providing non-speech auditory output to a user and irritating him with a continuous racket?
• One of the problems with audio that makes it more difficult to use than graphics is the problem with persistence. With audio, a short noise can be missed or miss-heard, or difficult to pick out in a succession of audio cues; with a long noise, it can be very aggravating to the user (the example of playing different sounds when you type '('s in an editor is an example). What are other differences in the way sound and graphics differ in the way they can be used?
• The DiGiano&Baecker paper discussed the multi-dimensionality of sound in terms of loudness, pitch, vibrato, rate of modulation, timbre, and tempo. While there must be many studies in ergonomics relating to audio, none of the papers really referred to the use of any of these studies for using audio in programs. En those programs that implement sound without the use of such a study, has there been any negative impacts?
• The DiGiano&Baecker paper claims sound adds another dimension of info available to the user and is important because there is a limit to the amount of visual info a user can process before becoming overloaded. Is it a maximum to the visual info or just a max to the total info - in which case changing the form from visual to audio doesn't allow the user to process anything more than before?
• It seems that visual info has two big advantages over audio:
  • the user can view many visual icons in a window at once, but wouldn't be able to listen to many at once - they'd have to click on each one separately.
  • visual images are better able to explain a function than sounds (such as a paint bucket icon or a trash can icon).
  The DiGiano&Baecker paper proposes "earcons" instead of icons - but don't address either of these issues. Can earcons overcome these problems?
• What kinds of events or sounds would be important when scanning (visually) through a C program?
• How can you ensure that a sound used for one event (say a crash on a hashing collision) doesn't overpower every other sound in the auralization? It seems easier for sounds to overpower other sounds than for a single color to dominate a view and distract from other colors.
• Sound helps visualization sometimes, but most of the time it doesn't convey the information effectively. Can we find out under which condition sound is effective so it can be used appropriately?
• Emergent sound patterns are useful for characterizing programs' behavior. How much training is needed to use this tool effectively. Is the pattern stable across different programs?
• As obvious as this question is, it must be made. Use of sound in algorithm animation seems to have an extremely narrow scope, almost no useful application. Have empirical studies shown that the use of sound is significant in expressing algorithm?
• It is interesting that both the papers suggest that specific sounds correspond to specific action types such as an execution sound or exception marker. Considering the observation that "computers have beeped at users for years" it seems that sound has its place in a useful interface (utilizing all forms of communication). In IRIS's latest version, an action sound is played when an application is initiated. What other sound cues can/should be expected in our usual work environment to take advantage of this work?
• What sort of problems might arise in the working environment from computers making lots of sounds of varying volumes? What about confusion arising from multiple computers (e.g. in a lab) all making similar sounds?
• In the Program Auralization paper, they talk about using sounds when reviewing code. How does the program know what part of the pages of code which can be viewed on a screen the user is focusing on? People usually move the cursor *away* from what they're reading.
• Both papers seem to simplify things by treating sound in discrete units: notes. Attributes of the notes such as frequency, timbre, or whatever are determined by some aspect of the program being auralized. This parallels the "animation" axis in the taxonomies we have looked at, but is very much on the low end of the axis. It is possible to incrementally change a sound's parameters, but neither of these papers has done this experiment. When would it be useful?
• The DiGiano&Baecker paper attacks syntax-directed editors (page 49), and proposes sound as an alternative to automatic formatting. Maybe I'm different, but I find automatic formatting to be very helpful. And since I use emacs, I have complete control over the formatting so I can set up my own formatting style. When I code I frequently jump around, leaving some statements incomplete to come back to later, so a persistent sound signifying such cases would be a nuisance. Would anyone out there care to defend syntax-directed sonification?