CS/PSYC 6750:
 Homework 2 

(Fall 2004, Section B)
Homework 2 - Cognitive Walkthroughs and Comparison of Web Sites
(Start 14 September; Due 23 September at beginning of class)

Note: This homework will also be done in pairs (teams of two 6750 students). However, each student will define a separate detailed task and will submit a separate report.

The purpose of this assignment is to give you practical skills in the important area of "expert usability analysis". These techniques can be used to evaluate either an existing design, or an uncompleted prototype, and do not require the participation of real users. For the purposes of this assignment you will be evaluating existing Web sites, but this technique can also certainly be used as you evaluate your projects, throughout the design process.

First, make sure that you have read DFAB (3rd ed.) Chapter 9, the class notes, and other relevant sources. Your assignment is to compare two web sites (www.delta.com and www.aa.com) for the specific task of checking whether a flight is on time, and at which gate it is arriving (your friend is arriving, and you are picking him/her up at the airport). You will conduct a cognitive walkthrough of each site, given that task, and then compare the sites for usability, based on the results of your analysis.

You will find a partner (preferably a different one from your first homework), and each will serve as the "expert" for the other. Note that in the steps below, you should make sure that your two tasks are slightly different. You will each hand in separate reports.

As described in DFAB, you need to:

  1. Note that a description of the system is usually required at the beginning, but in this case you can skip that step, since we are all dealing with the same two sites.
  2. Describe the task that the users will perform. Use the task described above as a basic guide, but you need to define the task in a bit more detail. For example, does the user know the (departure/arrival?) cities, the flight number(s), the (departure/arrival?) time, or some combination of these? What is typical for real users? Each student in a pair should define a separate task. For example, one student might assume the user knows the flight number and date, but not the timeor departure city; the other student might assume that the person knows the departure city and approximate arrival time.
  3. List the steps required to complete the task with each system.
  4. Indicate who the users are, and what kind of experience and knowledge you (the expert evaluators) can assume about them.
  5. Observe and document the interaction while an expert in usability (your partner) performs a cognitive walkthrough of the two sites.
  6. Create a summary report on the walkthrough for each site separately.
  7. Compare the usability of the two sites (for this task).
  8. Explain what you have learned by doing this assignment, and in particular any surprises.

What to hand in:
You should hand a report that covers each of these steps, as well as the raw materials and raw data sheets you used to gather the data.

Tell me again, where do we get the experts?!!
You will all serve as the usability expert for each other. Note that you can get any student currently in either of the 6750 classes to be your expert. You will work in pairs (i.e., be each other's expert), but you will still need to each hand in separate reports.