Evaluation Assignment
Evaluate a recent version of the Classroom 2000 web notes interface in terms of usability and usefulness. Consider the effectiveness of the interface given the typical user population and user tasks associated with the notes.

I have chosen to execute a rather informal Heuristic evaluation. This is being done with myself as the only evaluator. I've chosen the following four heuristics (from the larger list of ten) as guidelines:

• Visibility of system status
  The system should always indicate activity (or lack thereof) appropriately.
• Match between system and the real world
  Using real world terms - and information appearing in a natural and logical order.
• Recognition rather than recall
  All options and actions should be immediately visible.
• Flexibility and efficiency of use
  Mostly experienced users will access the system - tailor it to their needs and access methods.

The Evaluation Setting
The evaluation was performed in my dorm room, with a campus internet connection. I used Netscape Navigator 3.01 for a 68K Macintosh (not a PPC system). I used the high bandwidth link for November 20th's lecture on UbiComp. The low bandwidth version is identical with the exception of loading only one slide at a time into the slide window.

Included below are a the results of the evaluation process. Noted problems and suggestions are ranked in order of perceived importance while general observations are unranked.

Observations
• browser does indicate loading progress
• real player does indicate playing progress
• mouse passover reveals that thin timeline is a continuous link that can be clicked
  not immediately obvious, but easy to discover
• can click anywhere on the thin black timeline but not on the thick black timeline
• position of slide marker on timeline corresponds to time the slide was accessed during the lecture
• click on a web link and it opens up a new browser window
• all-slides.ps is a file that can be saved and printed out
• did not print out postscript slide example
• slide window and timeline window are not scroll-linked
• browser does allow resizing of timline/slide window separator
• cs6751 link takes you to the class web pages
• to position slides, scroll or click on timeline link
• to position timeline - only scrolling is possible

Problems
1. no contextual information in the timeline window
   Only slide numbers are given, unlike web links which have a title
2. can't choose desired media or combination of media
   If I don't want to load any slides, I can't choose that as an option
3. the visual icon of the "page with lines of text" does not match with the term "slide"
4. can't play audio from a slide that is unmarked
   can't trigger audio from a slide that has no ink on it - there are examples where a slide is visited during the course of the lecture, but was not marked
5. Just after 0:45 minutes - an unnamed web link is present
6. no link between broswer and real player
   The browser doesn't flip through slides as the audio plays through the lecture
7. top of left scroll window is not clearly outlined
8. takes too long to load
9. slides unnecessarily large

Suggestions
1. searchable audio and text transcript
2. incorporate video
3. add content headers or cues to the timeline slide markers so that you don't have to load the slides at all but can just bring them up individually, as needed. (like the web pages)
4. integrate AV presentation
   slides should change with audio and video.
   • advance by slide should automatically move AV and timeline (all navigation features should be tied together this way)
   • If possible, maintain time stamp info on the liveboard strokes and animate the strokes as they are made in time. Other simpler possibilities include: just display the mark as it is made, or highlight the mark as it is made
5. Optionally display each form of media
   only display lecture transcript with timeline annotations, or only slides, or only audio, or desired combinations, etc.
6. search to full text
   offer best matches to search string in small text snippits of the lecture, with options for showing slide, video or audio if one text phrase seems to be correct.
7. denote subsequent revisitings of various media
   some indicator (color) that this is a second or subsequent visit to a slide or web link
8. indicator on timeline for current listening position
9. Add audience audio

Rationale
While the current implementation is functional and somewhat usefull, the most helpful addition would be a full transcript of the lecture that contains pointers to all of the various media recorded during the course of the lecture. My one caveat to this suggestion is that I have no idea what such a transcript might look like - it may end up being rather unusable depending on how it appears visually and how easily it can be searched. My second primary sugestion involves a tighter integration of the various media. I think we (students) can easily figure out when the lecture moves from one slide to the next but things may get a bit more complex as you (teacher) become more familiar with having multiple past slides up and referencing those during the course of the lecture. Finally, the searchability of these media is key. Again it would be helpful to have some idea of how the system is currently being used. I would guess that there are a few "missed class" uses, while most of them being reference lookups. In that case, being able to search through the lectures would be most helpful.

Miscellaneous notes
I used the notes retrieval system once to find a reference that I needed to study for the midterm. Otherwise, I missed two class periods during the course of the semester and did not have need to revisit the notes. I don't particularly care for the system in general. It is too slow in getting started on my machine for me to use it casually, and requires that I have every other application closed. In other words, I would not use the system unless I had a particular need to. I would probably refer to my own notes first before looking to the online version. Often the meat of the lecture is not reflected on the slides - the alternate explainations and examples that enrich my learning. It does however, provide an excellent outline of the material presented during the course.

Kevin C. Scott mail web
Original HW#1