Dave Deitrich
Project Group N
Dave Deitrich
Walter Patterson
Byron Warner
INTRODUCTION
An important part in the design of any of any interface is the ability to measure the usability of the design. If you cannot measure the usability of the design, then you cannot predict how successful the design will be and identify possible areas for improvement. In this paper we describe strategies for measuring and improving the usability of our ATM interface design. We identify the important usability characteristics for our design, and then determine how to quantitatively measure those characteristics. We then provide a usability tracking table for our design which describes the initial, best, worst, and acceptable values for our attributes, as well as an overall usability measure for our design. Finally we describe the test subjects we would need to use in these studies, as well as possible confounding variables that we may face in our studies.
PART 1: IMPORTANT USABILITY CHARACTERISTICS
There are seven characteristics of our ATM interface design that make it an extremely usable design. They are (in order from most important to lesser importance) ease of learning, easy to remember, error free, safety/security, access, pleasant to use, and efficiency. In the paragraphs below we define each characteristics in terms of our design and describe why we feel they are important.
1. Easy to Learn
Since ATM machines are used by such a wide variety of people of all different professions and educational backgrounds, it is very important that average users can use our ATM design with no prior experience or training. The design must be simple and intuitive enough to be used by people who have little or no experience with computers or even older ATM designs. The menu designs, icons, and additional on-screen explanation text for our interface are designed to provide simple choices to the user at all times and clearly explain the meaning of each choice the user makes. If the icons and explanation text are not enough to give the user a clear idea of the meaning of a question, he can touch a constantly-present help button at any time to bring up additional information. The clear presentation of information and availability of online help should make our interface design extremely easy to learn by any potential user from any background.
2. Easy to Remember
Studies earlier in our project showed that average users use ATMs infrequently, perhaps as little as once or twice a month. Therefore it is important to make sure that users do not have to relearn how to use the ATM interface every time they use it. The menus for our ATM interface have been arranged in such a way that the most frequently used functions require the fewest presses to accomplish, thus reducing the amount of knowledge the user must remember when using the ATM. For example, to make a withdrawal requires the user to press only two buttons ("WITHDRAWAL" and the account to withdraw from; usually "CHECKING" or "SAVINGS") and the amount to withdraw. The user need only remember the words "WITHDRAWAL" and "CHECKING" (or "SAVINGS") in order to remember how to use the ATM again. In addition, the use of icons and other graphical displays help to "prompt" the user into remembering the function of buttons he has seen and used before. The low number of steps involved in most transactions and the constant reminders present help to make our interface design extremely easy to remember.
3. Error Free
Since management of money is so important to most people, it is critical that our ATM interface design try to reduce the number of errors that users make. Besides the constant availability of online help to aid the user, we have implemented several safeguards to help verify the user's intent and make sure that few mistakes are made. One safeguard is the use of icons to display previous choices made by the user, such as when transferring money between accounts. When selecting accounts to transfer money from and to, the icons for those accounts are displayed with arrows between them, showing the way that money will be transferred. This allows the user to constantly see where he is in the process and notice if any mistakes are made. Should the user notice a mistake, he can touch a constantly-present CANCEL button to abort the current action at any time. In addition, warning messages have been included in our interface to warn the user when he is about to make a potentially harmful transaction, such as withdrawing too much money from one of his accounts. Once a fiscal error is made, it is hard to correct, so the safeguards present in our interface help to prevent errors before they occur.
4. Safety/Security
The high rate of crimes that involve ATM users as victims is an unfortunate fact of life. Since most ATM machines are unmanned, located in easily visible and accessible areas, and available at all hours of the day and night, it is easy for criminals to stalk victims and rob them as they use the ATMs. We have attempted to reduce the potential risk to ATM users by redesigning the entire environment for ATM machines with increased safety in mind. We have provided additional measures such as lockable doors, auto-dial telephones for contacting local police, and security cameras to increase the safety for users while using the ATM. Making the user more safe and secure with his environment allows him to devote more concentration to the interface and thus increases its usability.
5. Accessibility
Since ATM machines are used by almost everyone in today's society, it is important to consider ALL types of users when designing the interface, including handicapped ones. We have been very careful to include many features to help aid physically challenged users when using our ATM design. The ATM environment is large enough to allow people with wheelchairs or other mobility aids to easily gain access to the ATM. The dramatic visual display allows full functionality for deaf people, who could not hear audio information if provided. However, audio input and output is also provided for blind persons who could not see the visual displays. All handicaps of a user would be encoded on the user's banking card, so the machine would automatically be aware of any potential problems or difficulties and automatically take steps to minimize these difficulties (such as turning on audio input/output if a blind person inserts his card). These steps greatly increase the usability of our design for physically handicapped users.
6. Pleasant to Use
If users like the look and feel of an interface, they will probably experience less stress when using it, which in turn makes the design of the interface more usable. Our ATM interface uses color graphics and sharp text displayed on a high resolution monitor, which provides a much more dramatic and colorful presentation than today's text-based ATM interfaces. In addition, "cute" icon designs and display tricks (such as the use of animation and other special effects) can help increase user enjoyment, even to the point of making ATM machines fun to use. Since all aspects of the display are controlled by software, the cost of including these "bells and whistles" in the interface is negligible.
7. Efficient
Although ATMs are generally used infrequently by people, there are certain times and situations when the number of users wishing to use an ATM outnumber the number of ATMs available. Good examples of this are ATM machines in large shopping malls, or ATM machines at local banks after 5 PM on paydays. In these situations it is important that our ATM interface be efficient and quick so as to reduce the amount of time a user spends on a transaction, allowing more people to use an ATM machine in a shorter period of time. As we mentioned earlier, the menus for our interface have been designed so that commonly-performed tasks take the fewest amount of touches and selections. Therefore, most users will only have to perform a quick and minimal series of steps to complete their transaction. In addition, since it is easy for users to remember how to use the ATM, they do not waste time figuring out how to use the interface each time they use it. These features make our ATM interface quick and efficient, thus increasing the throughput of each machine during peak usage.
PART 2: TRACKING OF USABILITY ATTRIBUTES
In order to improve the usability of our interface, we must be able to "track" the usability characteristics of our system in order to detect problem areas. Fortunately, all of our interface's characteristics can be easily tracked. Below we list each usability attribute and how it is measured. We also include some suggested benchmark tests for measuring all attributes.
Easy to Learn
This attribute can be tracked by recording the average time it takes novice/beginner users to complete a task and then compare it to the average amount of time it takes expert users to complete the same task. Hopefully if our interface is as easy to learn as we think, then the difference in times will be fairly small. By dividing the average "novice" time by the average "expert" time, we will get a ratio that is proportional to how much extra time a novice requires to learn the system. The closer to 1 this ratio is, the easier our interface design is to learn by novice users on average.
Possible Benchmarks:
* Log onto system; Withdraw $20 from checking account; Log off system. * Log onto system; Deposit $400 into savings account; Withdraw $50 from checking account; Log off system. * Log onto system; Transfer $200 from savings account to checking account; Print listing of all transactions for current month; Cancel check #104, Log off system. * Log onto system; Print listing of last 10 checks cashed; Order new checks starting at #500; Withdraw $100 from savings account; Log off system.
Easy to Remember
This attribute is also tracked by recording time to complete a task on average. A good usability test for this attribute would be to time how long an experienced (but not necessarily expert) user takes to complete a task, and then ask the same user to perform the same task a month later (with no prior practice for the second performance). If our interface design is easy to remember then the time for the second performance should be close to or better than the time for the first performance. Again, this could be measured as a ratio of second performance time to first performance time, with a result close to or less than 1 meaning that our interface design is easy to remember.
Possible Benchmarks:
* Log onto system; Withdraw $20 from checking account; Log off system. * Log onto system; Deposit $400 into savings account; Withdraw $50 from checking account; Log off system. * Log onto system; Transfer $200 from savings account to checking account; Withdraw $100 from savings account; Log off system.
Error Free
This is probably best measured by recording the average number of errors made by users when performing a set of tasks on our ATM interface. Ideally, this number should be very close to, if not equal to, zero.
Possible Benchmarks:
* Log onto system; Withdraw $20 from checking account; Log off system. * Log onto system; Deposit $400 into savings account; Withdraw $50 from checking account; Log off system. * Log onto system; Transfer $200 from savings account to checking account; Print listing of all transactions for current month; Cancel check #104, Log off system. * Log onto system; Print listing of last 10 checks cashed; Order new checks starting at #500; Withdraw $100 from savings account; Log off system.
Pleasant to Use
This attribute is more subjective than the others dealt with so far, so it cannot easily be measured by timings or observations. The best way to measure this attribute would be to ask the user how much "fun" he had using the interface in a questionnaire after a usability experiment. Each questionnaire answer would then be assigned a "enjoyment" rating from +2 to -2, where +2 means that the user loved the interface (compared to the standard function key ATM interface) and a -2 means the user hated the interface. The average of the ratings would be a fair estimate of how pleasant our interface is to the average user.
Possible Tests:
* Ask user to fill out questionnaire after completing any of the other usability tests.
Safety/Security
This attribute is harder to test than the others. The best way to measure it would probably be to put the ATM environment into actual service and compare the "victimization" rate for our environment to the rate for the standard ATM environment using police crime statistics. This is somewhat grisly, however, and is impractical from a prototyping standpoint. Therefore the best way to measure this might be to have experts in the field of safety and security inspect full-scale models of our environment design and give comments. Experts in safety would include police officers and investigators, private security consultants, and other people involved with law enforcement. These people could be given questionnaires which would then be measured for satisfaction similar to the questionnaires for "pleasantness". A rating of +2 on a questionnaire would mean that the expert considered the environment very safe (compared to the standard ATM environment) and a rating of -2 would mean that the expert considered the environment unsafe. An average would again be used to estimate the safety of our design.
Possible Tests:
* Allow experts to inspect full-scale model of ATM environment for as long as they wish. * Ask them to fill out questionnaire after completing inspection.
Accessibility
Again, this attribute is very subjective. We would again have to ask "experts" to give us their opinions on how accessible they feel our environment is to handicapped persons. In this case the "experts" would be physically-challenged test subjects. They would be asked to perform a series of tasks on our interface and then given a questionnaire to determine how much difficulty they had performing the tasks. It would probably be best to divide the results by handicap type, since blindness is a much different problem than being in a wheelchair, for example. By measuring and averaging the satisfaction of handicapped persons, we would get an indication of how usable our design is to average handicapped persons.
Possible Tests:
* Ask handicapped user to fill out questionnaire after completing any of the other usability tests.
Efficient
This is perhaps the easiest attribute to measure. Since ATMs are used by persons of all experience levels and backgrounds, we could simply take the average time to perform a task for each user (regardless of circumstances) and then average them together to get the "average time" it takes the "average user" to perform the "average task". The only difficulty would be to make sure that our testing population is diverse enough not to favor one group of users in the results. We would then try to get this measurement as low as possible.
Possible Benchmarks:
* Log onto system; Withdraw $20 from checking account; Log off system. * Log onto system; Deposit $400 into savings account; Withdraw $50 from checking account; Log off system. * Log onto system; Transfer $200 from savings account to checking account; Print listing of all transactions for current month; Cancel check #104, Log off system. * Log onto system; Print listing of last 10 checks cashed; Order new checks starting at #500; Withdraw $100 from savings account; Log off system.
PART 3: USABILITY TRACKING TABLE
==================================================================================
USABILITY MEASURING VALUE TO BE CURRENT WORST PLAN BEST
ATTRIBUTE INSTRUMENT MEASURED LEVEL LEVEL TARGET LEVEL
==================================================================================
Easy to Benchmark tasks Ratio of time for 1.3 1.5 1.1 1.0
Learn for typical NOVICE to complete
transactions task vs. time for
EXPERT to complete
task
----------------------------------------------------------------------------------
Easy to Benchmark tasks Ratio of time for 1.2 1.2 0.1 0.75
Remember for typical RETURNING USER to
transactions complete task vs.
time for EXPERI-
ENCED USER to
complete task
----------------------------------------------------------------------------------
Error Benchmark tasks AVERAGE NUMBER of 0.75 0.5 0.1 0.01
Free for typical observed errors errors/ errors/ errors/ errors/
transactions per task task task task task
----------------------------------------------------------------------------------
Safety Questionnaire Satisfaction +0.5 +0.8 +1.5 +2.0
following rating from +2
inspection (good) to -2
(bad)
----------------------------------------------------------------------------------
Access- Questionnaire Satisfaction +1.0 +1.5 +2.0 +2.0
ibility following rating from +2
experiment (good) to -2
(bad)
----------------------------------------------------------------------------------
Pleasant Questionnaire Satisfaction +0.75 0.0 +1.0 +2.0
to Use following rating from +2
experiment (good) to -2
(bad)
----------------------------------------------------------------------------------
Effici- Benchmark tasks AVERAGE TIME to 100 sec 100 sec 80 sec 60 sec
ency for typical complete a task
transactions (overall)
----------------------------------------------------------------------------------
PART 4: OVERALL USABILITY MEASUREMENT
Designing an overall usability measurement for an ATM machine is fairly difficult. ATMs are a public service provided by banks, and usually don't have much of an influence on the profits the banks make other than possibly attracting more customers to the bank. Therefore it is not really possible to measure an ATM design in terms of profit vs. cost. However, one factor we can use in calculating an overall measurement is that the usability characteristics we defined above are applicable to ALL ATM interface designs, and not just our own. Therefore performing the same tests outlined above on other ATM interfaces and then comparing them to our own interface would be a fairly good overall measurement for comparison between designs.
The values for the Easy to Learn, Easy to Remember, Error Free, and Efficiency attributes would have to be inverted before the overall calculation is made. This is because for those attributes a LOWER value is more desirable than a HIGHER. Thus if we invert them higher numbers will indicate better results than lower numbers for all attributes. Then the geometric mean could be taken of all attributes to come up with an overall usability measurement. Since geometric means treat all values as weighted more or less equally, it may be necessary to normalize all values to a standard scale so as to prevent falsified results generated by the different scales used.
PART 5: ITERATIVE IMPROVEMENTS/MODIFICATIONS
Once we have our usability measurements and methods of calculation, we can begin to improve our interface design based on the results of our tests. The values for Easy to Learn and Error Free would indicate how good our display design and online help are. We hope that the general design of our display will be enough to provide users with a good idea of the functions of each button and selection, and that the online help will be able to provide additional service if the general display is not enough. A low value in either of these attributes could be improved by redesigning the graphics and icons in our menu designs, and making the online help clearer. Improving the icon and display appearance to communicate more information to the user may also improve the rating for the Easy to Remember attribute.
The Safety and Accessibility questionnaire results will hopefully point out potential flaws in our environment design, which could then be corrected by redesigning the environment. A successful improvement will be reflected by a higher average score for these attributes. Improving the values of the Pleasant to Use attribute is more difficult, since people often have a hard time defining how to make something more "pleasant". The best action in this case would probably be to address any problems/annoyances pointed out by the test subjects on the questionnaires. We do not want to add too many bells and whistles to the interface or we may start distracting the user and interfering with task performance. Again, a successful modification or elimination of an annoyance should be indicated by an increase in the average satisfaction rating.
A low Efficiency rating may indicate that we have incorrectly estimated which tasks are performed most often by the average user. This could be corrected by modifying the tree structure of the menu design to allow more common transactions to be performed with less steps. By subdividing the Efficiency rating into ratings for separate tasks/benchmarks, we can more easily pinpoint the problem area in the tree and then modify that part without affecting any parts of the tree that seem to work well.
PART 6: TEST SUBJECT DESCRIPTION
Since there are no common attributes to average ATM users, it is very important to get a very diverse test population for our usability experiments. The test subjects must vary in age, experience, educational background, and technical background. The test population will need to be divided into several groups including novice users, expert users, first-time users, experienced users, safety experts, and handicapped persons. These groupings are important for certain tests, such as the Easy to Learn, Safety, and Accessibility tests. Test subjects can belong to multiple groups so as to reduce the testing costs and number of people necessary to conduct the tests. However, above all else it is important to get an extremely diverse test population since the targeted user population will also be very diverse.
PART 7: CONFOUNDING VARIABLES
One of the largest confounding variables will be to make sure we have a diverse enough testing population to get good testing results. It will probably be necessary to do an interview of all testing candidates and selectively choose a pool of users to serve as test subjects. This should hopefully reduce the possibility of bad data from a particularly biased or experienced group of test subjects.
Another confounding variable that may be a factor in some calculations is the novelty of a touch-screen interface. The touch-screen is so different from the standard function-key ATM interface that users may initially have a higher opinion of the interface because of the originality of the idea. However, as the users become more experienced and get more used to the interface they may notice problems or difficulties that they ignored during earlier tests. This would have a particularly bad affect on the accuracy of the Pleasant to Use attribute calculations. The best way to deal with this problem may be to not give the questionnaire for pleasantness to test subjects until they have used the interface several times, so that they have a better feel for the interface and are not as biased by the newness of the design.
CONCLUSION
In this paper we have described strategies for measuring and improving the usability of our ATM interface design. By identifying the important usability characteristics for our design and determining how to quantitatively measure and improve them, we can systematically "work the bugs" out of our design to improve the interface. The current interface design for the ATM machine has been unchanged since its development 15 years ago. With today's technology, we can design a much improved interface that is easier, more reliable, safer, and more enjoyable to use than the current design. However, up until now our project has consisted mostly of speculation and brainstorming with no real proof. The usability tests outlined in this paper would enable us to provide positive numerical proof that our design is better than the current interface design.