Project

This term you will undertake a group project to accomplish the following goals:

  • evaluate some computing-related task or problem
  • develop interface design alternatives for the task or problem
  • implement a prototype of your design
  • evaluate your design

This project should provide you with hands-on experience with the tasks that interface designers face every day. Ideally, the topic of the project will be a problem that matters to some "real-life" people. These people then will serve as your "clients", with whom you must communicate and from whom you will learn about their tasks and problems. The instructors will provide a list of possible projects for you to consider. You are also free to develop a topic on your own, but you must get it approved a priori. It is your responsibility to contact the clients for your project, as they will not come to you.

Each project group will be graded as a team, i.e., each member of a group member will receive the same grade. If necessary, instructors will poll team members to check whether each member is making a proper contribution. Lack of participation by any individual may precipitate a grade reduction for that individual. Within the team, you must negotiate on how much and what each person will contribute. Think carefully about your team members: Where do people live and what hours do they work? Where will you meet? What skills do the different individuals bring to the group (computing, programming, design, evaluation, statistics, etc.)? We strongly encourage you to form a heterogeneous composed of individuals possessing varying skills.

Project Deliverables

For each part of the project, each group must submit a report. This report must be in the form of a physical document; we will not accept electronic copies of your report. This requirement varies from previous versions of this class, so take special care to prepare TWO "hard copies" of your report for submission.

Each report must follow a defined structure. Details about the structure are provided below. The reason that we enforce this structure is that grading criteria will be based directly on the content that is defined by the structure. Feel free to add to the structure when you have additional necessary, relevant, or interesting content, but do not alter the order of the structure.

As with any written report, in addition to grading the document based on content, we will also be grading based on degree of professional preparation, expressiveness, grammatical soundness, and the ease with which it can be viewed and understood. A good design effort can easily be hampered by a poor communication of what was done. Make sure that you produce a report that is illustrative of your efforts and process.

Finally, late submissions will not be accepted. You must meet the deadlines stated in the class schedule on the class web page. The physical reports are due just prior to the time that the instructor leaves the classroom at the conclusion of the day's lecture. We strongly recommend that you finish the printing of your final report on the day prior to the due date. You have been forewarned.

Part 0 - Topic Definition

Due January 18

Part 0 involves forming your team and picking your project topic. As stated previously, aim to assemble a group of people with different skills, such as programmers, designers, psychologists, and adept writers. With regard to this last attribute, we recommend that each team have at least one native speaker of English who can write or proofread the written report.

We will use a class swiki to keep everyone aware of the different project groups. There are two primary swiki pages of interest here: (1) a "swap meet" page where as-of-yet incomplete groups can write requests for additional group members and individuals and advertise their skills and (2) a "project group" page where each group will list group name, group member names, and a brief description of what the group will be working. Groups should also feel free to employ the swiki for group communication and brainstorming.

There are two deliverables for Part 0:

  • Submit a one-page paper with a team name, members, and topic.
  • Place the same information on the class swiki

Part 1 - Understanding the Problem

Due February 8

The key goal of this first substantive part of the project is to deeply understand the problem space that you are addressing, its set of pertinent users, and the issues and constraints that are involved in the problem. If the task is accomplished through an existing system or interface, you should perform an interpretive evaluation of that system or interface to help you learn more about it. The most important goal of Part 1 is to identify important characteristics of the problem that will influence your subsequent design. A major mistake that students make on Part 1 is to suggest potential solutions without first identifying the problem and its characteristics. You'll have plenty of time for designs of possible solutions in Part 2. For now, suppress the urge to problem-solve and concentrate your efforts fully on developing an in-depth understanding of the problem at hand.

In class we will discuss different techniques for acquiring this kind of information. Feel free to utilize the techniques that you feel are most appropriate to the particular task you are examining. In fact, it is wise to include in your report a brief justification of why you selected certain techniques as well as why you rejected others. Your report and deliverable for this part should deeply examine the problem of study. In general you should be attempting to answer these questions:

  • Who are the potential users?
  • What tasks do they seek to perform?
  • What functionality should any system provide to these users?
  • What constraints will be placed on your eventual design?
  • What criteria should be used to judge if your design is a success or not?

Use the following structure for your report. Remember to state how you collected your data and justify the methods that you used. If you selected one method over other possible methods, include a brief statement of why you chose not to use those other methods.

  • [  5 pts] An overview of the problem and a statement of why an interface or system is necessary to solve it.
  • [15 pts] A description of the important characteristics of the users of the system.
  • [30 pts] A task analysis consisting of the following items.
    • [10] A description of the important characteristics of the tasks performed by users.
    • [10] A description of important characteristics of the task environment.
    • [10] A simple structured task analysis of the problem in one of the forms described in the textbook.
  • [15 pts] An analysis of the existing system, automated or manual, including its strong points and deficiencies.
  • [15 pts] A description of the larger social and technical system in which your design will intersect.
  • [15 pts] An initial list of usability critieria, or principles, that should be used in the eventual evaluation of your design. Include a high-level description of how you could measure the successful adherence to these principles.
  • [10 pts] A discussion of the implications of what you learned above. Go beyond the usability criteria in this section.

The last item in the list above is critical. Don't only describe the target users, tasks, environment, etc. You must also tell us how these attributes should or will influence your eventual designs. Are there any implications to be made from the user profiles and other data you learned? We will be very careful to look for this information in your report.

Part 2 - Design Alternatives

Due March 1

The key goal of Part 2 of the project is to use the knowledge gained in Part 1, as well as that from class, to develop multiple design alternatives for your problem. This is the stage of "informed brainstorming." These alternatives should explore the design space of the problem.

In this part of the project you will develop mock-ups, storyboards, and sketches of your interface designs. That is, you should provide pencil-and-paper or electronic images of the interface at various stages. You do not need to build a working prototype. In fact, we recommend that you do not try to develop full prototypes in this part so that you can focus your time and effort on a broad exploration of the many design possibilities that exist for your problem or task.

Although we are not looking for a full-scale prototype, your design sketches should be sufficiently detailed for a potential user to provide useful feedback about the design. Along with your design mock-ups, you should provide a brief narrative walk-through of how the proposed system will work. Perhaps most importantly, you should also include your justifications for why design decisions were made, and what you consider to be the relative strengths and weaknesses of your different designs.

The design process you follow here is important. Recall the video on the IDEO design team and how they accomplished group brainstorming. You should arrive at your different designs through direct collaboration and group brainstorming. Do not split up, have everyone create one design, and present each person's design as a possible alternative. Good, creative design processes do not work in this fashion. Your results should come from something more like a brainstorming session with all team members present. You should seek to create some fundamentally different design ideas, i.e., concepts all over the potential design space for the problem you have chosen. The key is to push the boundaries of the space of design possibilities.

Your project report should include all the explanatory material mentioned above as well as all the design sketches, drafts, storyboards, etc., that you generated. Make sure that your report adequately reflects the design process that your group undertook. The key in this part of the project is to develop several different design ideas, not just a set of minute varations on some basic design. At a minimum, you must submit three different designs. It cannot be stressed enough that we seek significantly different design ideas; quality is more important than quantity. In particular, we would much rather see three very different designs descibed in great detail than five or six rather similar designs described in shallow detail.

Use the following structure for your report.

  • [  1 pt] Project Description: Write an updated one paragraph description of your project. Simply re-introduce the general area of application, intended tasks it will support and the intended user population.
  • [  4 pts] Requirements Summary: Briefly state key requirements from your system. Again, the goal here is to re-introduce the requirements developed in Part 1, though it is OK if you introduce new or altered requirements here. Do not exceed one page in this summary.
  • [15 pts] Design Space: Describe the design space of the potential interfaces for your system. In particular, answer the following questions (you may use each of these questions as section sub-headings if you wish, but that is not required).
    • What requirements may be difficult to realize?
    • What are some tradeoffs that you should or did explore?
    • Which tasks will be easiest to support? Which are hardest?
  • [15 pts] Design Summary: Briefly describe the design alternatives that you considered exploring, including alternatives that you did not ultimately pursue. Do not cover every idea that you discarded, but rather group them and discuss as a whole. Make sure to justify your choices (Why did you not pursue certain avenues? Why did you decide to pursue the designs that you actually produced?). Justifications need not be lengthy; a few sentences for each should suffice.
  • [60 pts] The designs: Present each design that you created. Remember that you should present at least three designs. Cover each design in its own section by presenting the following information.
    • [10%] A brief overview of the design.
    • [35%] Illustrations of the design (sketches, storyboards, etc.)
    • [20%] At least one scenario written from a user's perspective.
    • [35%] An assessment of this design (advantages, disadvantages, and the degree to which your requirements can be met by the design). Include feedback from potential users in the assessment. Make sure to express how you gathered this feedback.
  • [10 pts] Requirements changes: You more than likely modified, added to, or removed elements of your requirements and usability criteria as a result of conducting the design process. Discuss these in this section... what were they and how did they arise?

In addition to producing the report, you will also have to create a poster that illustrates the problem and users that you are addressing, the requirements that you have developed, and the multiple design alternatives that you have developed. We will use two full class days as a poster session near the end of this part. Everyone will then circulate and interact with the designers. The idea here is that each group can use this opportunity to get feedback about their design ideas as they narrow their design space and head into Part 3 of the project.

Part 3 - System Prototype and Evaluation Plan

Due April 3

In Part 3 of the project, your group will implement a detailed prototype of your interface. You can use any prototyping tools that you would like to assist this process (such as VB, Hypercard, Director, PowerPoint, web pages, clay, paper, plastic, etc.). You should be able to get much of the interface functionality working, but clearly you will not be able to implement all back-end application functionality. The key goal here is to develop as fully as possible the interface, not the entire system. We will discuss in class the different forms that this can take. Note that you should feel free to "mix and match" aspects of the different designs from Part 2 into the Part 3 prototype.

You must provide a set of initial usability specifications for your system and a plan for an evaluation of it. To develop usability specifications, consider the objectives of your design. For example, if you are working on a calendar manager, you might specify time limits in which you expect a user to be able to schedule or modify an appointment, or a maximum number of errors that you expect to occur. Basically, you should list a set of criteria by which your interface can be evaluated.

You should also include an initial evaluation plan for the system. What kinds of benchmark tasks would you have users perform to help evaluate the interface? What kind of subjective questionnaire would you deploy to have a user critique the interface? You will actually conduct some of this evaluation in Part 4, so you should do your best to set it up now. The key here is not to do some exhaustive description of a usability evaluation plan, but to motivate why the particular plan you propose is appropriate for this interface.

Note that developing an initial evaluation plan is also a good way to figure out how much of the interface you need to develop. You should be able to build and connect enough of the application functionality to be able to conduct an initial usability evaluation with the benchmark tasks as you are proposing here.

Your report write-up for this part should include a description of your system prototype. You can include screen shots or photographs to help explain it and text to describe how a user would interact with it. Discuss the implementation challenges you faced. Were there aspects that you wanted to build but could not? In addition to the prototype description, it is key to include a justification of why you built your prototype. What's special about this particular design with respect your problem? You are encouraged to include feedback from users and from the poster session in your justification.

The report for this part also must include the usability specifications that you established and a description of the evaluation that you are planning. The plan does not have to be presented in great detail since the actual evaluation will occur in Part 4. We will try to give you helpful feedback about your plan here to assist with the testing in Part 4.

Use the following structure for your report.

  • [  1 pt] Project Description: Write an updated one paragraph description of your project. Simply re-introduce the general area of application, intended tasks it will support and the intended user population.
  • [  4 pts] Requirements Summary: Briefly state key requirements from your system. Again, the goal here is to re-introduce the requirements developed in Parts 1 and 2, though it is OK if you introduce new or altered requirements here. Do not exceed one page in this summary.
  • [75 pts] Prototype Description:
    • [  5 pts] An overview of the prototype that you developed.
    • [20 pts] Each piece of the prototype in more detail, using screen shots or photographs to help illustrate the design.
    • [10 pts] At least one scenario from a user's perspective.
    • [20 pts] Rationale: why did you choose this prototype? What are its advantages and disadvantages with respect to your requirements and to your ability to evaluate it?
    • [20 pts] Changes to your requirements: did you alter your requirements or usability criteria while developing your prototype? If so, what are they and how did they come about?
  • [25 pts] Initial Evaluation Plan: Discuss usability criteria and requirements that your prototype can address and how you plan to address them. In what ways do you plan to measure the effectiveness of your interface? Make sure to cover the three techniques that you plan to use.

Each group will also give an in-person demonstration of their prototype to the instructor and TA.

Part 4 - Evaluation

Due April 27, 4pm

In the final part of the project, your group will conduct an evaluation of the prototype developed in Part 3 based on the plan given in Part 3. We expect that your evaluation will involve sample users interacting with your system. These users will likely be your client(s) and maybe other students from class or other people who would fit your target user population. Give the users a few simple benchmark tasks and have them interact with your interface. Closely study what occurs. Deploy a questionnaire to get their subjective feedback about the interface and interaction.

Use the following structure for your report.

  • [  1 pt] Project Description: Write an updated one paragraph description of your project. Simply re-introduce the general area of application, intended tasks it will support and the intended user population.
  • [  4 pts] Requirements Summary: Briefly state key requirements from your system. Again, the goal here is to re-introduce the requirements developed in Parts 1 and 2, though it is OK if you introduce new or altered requirements here. Do not exceed one page in this summary.
  • [15 pts] Overview: describe the the evaluation techniques, tasks, and users involved in your study. For each technique and task, include a rationale (why did you select these tasks and methods?)
  • [60 pts] For each technique that you used...
    • [25%] Data Presentation: simply present the collected data. Use your best judgment on how to summarize your data, if necessary.
    • [25%] Data Analysis: describe what your collected data mean with respect to your usability criteria and requirements.
    • [50%] Design Implications: Does your prototype need to be altered in order to address the results of the analysis, or was it completely successful? What improvements could be made to the design to address any shortcomings? Did you discover any major flaws that would suggest a completely different type of design?
  • [25 pts] Critique and summary: What were the advantages and disadvantages of your evaluation? What would you have done differently knowing what you know now (both design-wise and evaluation-wise)? Given more resources, what could you have done that would have produced significantly more insightful evaluation results (again, whether this is an improved prototype or a different evaluation path).

The key to this part of the project is not to simply describe your evaluation methodology but to rise above that and describe what you learned from it. Although you do not necessary have to directly address the following questions, keep them in mind as you conduct your evaluation and write your report:

  • Why did you chose the benchmark tasks that you did?
  • Why did you ask users what you asked?
  • What conclusions can you draw from the studies?
  • What aspects of your design "worked" and what failed to meet your specifications?
  • If you had more time to work on the design, what would you now change and improve?

Remember, no designer ever gets a system "just right." We will reward teams who honestly and carefully assess their design and who clearly provide a plan for its improvement.

Project Presentation

At the end of the semester, each group will present their project results to the class and, if possible, to their clients. Each group will be expected to give a professional summary and walk-through of their design and prototype. It is important that you do a good job communicating all your efforts for the semester. You want to make sure that your objectives in the project are discussed, and your system is clearly presented. Also describe what you learned from your usability study. In particular, address these areas:

  • Motivation - what problem are you addressing?
  • Top requirements - what did you learn from users?
  • Design - what does your solution look like?
  • Evaluation - what did you do and what were the results?
  • Conclusions - if you had more time, what would you do next?

Practice your presentation! Depending on your class section, you will have only 10 or 15 minutes. This is much shorter than you think. You absolutely must practice and keep the presentation under the allotted time period. There is simply not enough class time to give people more, and you will be forced to conclude your presentation abruptly if you attempt to exceed your allotted time period.