GOMS Analysis Techniques
removed removed
Final Essay
CS 6751 – Human Computer Interaction
Dr. G. Abowd
December 1997
Intro and Need for User Modeling
Designers and analysts need models. For the designer, generative
models contribute to the design process. Engineering formulas can
be thought of as generative models. A mechanical engineer may use
thermodynamic cycle equations (models) to determine the proper size of
a steam turbine. Analysts use evaluative models to provide comment on a
system. In metrology, one compares measured data from manufactured
artifacts to ideal geometric models in order to comment on part quality.
Models can be used in a similar fashion in the field of Human Computer
Interaction. Cognitive modeling attempts to algorithmically state
how a user will interact with a system. That is, a cognitive model
predicts user behavior. Clearly, having a reasonable method for estimating
a users internal processes and the effect of those processes on the users
interaction with a system would be an extremely valuable tool in program
and interface design.
This essay examines one particular technique for user modeling: GOMS.
GOMS is one of the few widely known theoretical concepts from the field
of human computer interaction.
Overview
GOMS, of course, is an acronym. The letters stand for four ideas
important to users’ internal processes: Goals, Operators, Methods and Selection
Rules. The concept was originated by Card, Moran and Newall in their
1983 book The Psychology of Human Computer Interaction. Like
most new concepts, there were many areas for improvement and others seized
on those research opportunities to develop several variants of the GOMS
method. The original method and the variations are discussed in turn.
For all of the variants, the definitions of the major concepts are the
same. Goals are the what the user has to accomplish.
An operator is an action performed in service of a goal. A
method is a sequence of operators that accomplish a goal.
There can be more than one method available to accomplish a goal.
If more than one exists, then one of them is chosen by some selection
rule. Selection rules are the mostly ignored in typical GOMS
analyses. There is some flexibility for the designers/analysts definition
of all of these entities. For instance, one person's operator may be another’s
goal. The level of granularity is adjusted to capture what the particular
evaluator is examining.
The plain, or ‘vanilla flavored,’ GOMS first introduced by Card, Moran
and Newall is now referred to as CMN-GOMS. Keystroke Level Modeling
(KLM) is the next GOMS technique and was also introduced by Card, Moran
and Newall in their 1983 book. This technique makes several simplifying
assumptions that make it really just a restricted version of GOMS.
The third major variant on the GOMS technique is the ‘Natural GOMS Language’
or NGOMSL for short. This technique gives a very strict, but natural,
language for building GOMS models. The final variation of GOMS to
be examined in this paper is CPM-GOMS. This technique is based on
the Model Human Processor. The main advantage of CPM-GOMS is that
it allows for the modelling of parallel information processing by the user.
KLM Variation
KLM is a simplified version of CMN-GOMS. It eliminates the goals,
methods and selection rules leaving only primitive operators. Only
six operations are allowed: 1) pressing a key, 2) moving the pointing device
to a specific location, 3) pointer drag movements, 4) mental
preparation, 5) moving hands to appropriate locations, and 6) waiting
for the computer to execute a command. This method does not allow
for either slips or mistakes. The placement of the keystrokes and
pointer operations are straight forward, but the placement of the mental
operations is not. Mental operations are usually placed by some heuristic
method. They are put before single commands or before conceptual
blocks of commands (for instance ctrl-c or double click). The times
for each of the six tasks has been empirically determined. Thus,
once the operations have been arranged into a stream, total task execution
time calculation is trivial.
The main reason a designer or analysts would use this technique is that
it is a very fast. Different designs or systems can be compared against
one another quickly. Further, it does not require that the evaluator
be an expert in GOMS because the procedure is a explicitly laid out recipe.
The major caution is that nature of the algorithm makes it nearly always
underestimate the execution time.
CMN-GOMS Variation
This is the ‘vanilla flavored’ GOMS originally introduced in Card, Moran
and Newell’s book. This technique requires a strict goal-method-operation-selection
rules structure. The structure is rigid enough that the evaluator
represents the tasks in a pseudo-code format (no formal syntax is dictated).
It also provides a guide for how to formulate selection goals. This
methods can also be used to estimate the load the tasks places on the user.
For instance, examining the number of levels down in a goal branch can
be used to estimate the memory demand on the user. The must remember
information about all of the levels above them. Card Moran and Newell
were able to achieve 90% correct prediction rate using their method.
This technique is more flexible than KLM because the pseudo-code is
in a general form. That is, it can be executed for different scenarios.
KLM’s procedure is a simple list that has to be recreated for each slightly
different task.
NGOMSL Variation
David Kieras developed the Natural GOMS Language technique in 1988.
The motivation was to make GOMS/CCT (cognitive complexity theory) simple
to use, similar to KLM, and still keep the power and flexibility of standard
GOMS. This was necessary because GOMS did not have very well defined
semantics. This lack of definition meant that two equally competent
evaluators could do evaluations on the same system and come up with very
different results. Kieras’ result was the development of high level
(natural language) syntax for GOMS representation with directions for going
about doing a GOMS evaluation. The recipe is referred to as a “top
down, breadth first” expansion. The users high level goals are unfolded
until only operators remain. Again, there are granularity considerations
here. Generally operators are considered to be keystroke level operations
but this is not a rigid requirement.
Since NGOMSL is based on CCT it has certain properties that make it
unique. CCT (this could be a whole other paper) not only gives estimations
for execution times, but also predicts learning time. NGOMSL inherits
this ability. It also, however, shares one of the major disadvantages
the all of the previous methods. NGOMSL models user interaction as
a serial operation. One operation occupies the user completely, there
is no multitasking. This makes NGOMSL inappropriate for tasks
where the users are under time pressure, highly practiced and, in reality,
do act in a parallel fashion.
CPM-GOMS Variation
This work was done by Bonnie John, a former student of Alan Newell.
She is one of the most influential researchers working in GOMS research.
CPM stands for two things: Cognitive Perceptual Model and the project
planning technique Critical Path Method. Because it borrows elements
of the critical path method, it is unique in that it does not make the
assumption that the users interaction is a serial process. The technique
is also based directly on the model human processor. Evaluators begin
a CPM-GOMS analysis in the same way they would a CMN-GOMS analysis.
However, when the tasks are broken down just to the level where they are
still perceptual or motor, the evaluator applies techniques from the model
human processor. The tasks are first joined together serially and
then examined to see which actions can be overlapped so that they happen
in parallel. This technique facilitates representation of overlapping
and very efficient ‘chunks’ of activity characteristic of expert users.
The estimated times by CPM-GOMS are generally faster since they do not
allocate as much time to the ‘prepare for action’ type operations.
This technique is the most difficult to implement. Therefore it
has the problem of discrepancies between evaluators. Research is
currently being conducted to improve the CPM-GOMS technique so that it
can be used without the evaluator having a high level understanding of
the GOMS theoretical foundations.
Weakness of GOMS Overall
All of the GOMS techniques provide valuable information, but they all also
have certain drawbacks. None of the techniques address user fatigue.
Over time users performance degrades simply because the user has been performing
the same task repetitively. The techniques are very explicate
about basic movement operations, but are generally less rigid with basic
cognitive actions. It is a fact that slips cannot be prevented, but
none of the GOMS models allow for any type of error. Further, all
of the techniques are only applicable to expert users, novice’s are left
out. Functionality of the system is not considered, only the usability.
If functionality were considered, the evaluation could make recommendations
as to which functions should be performed by the system (i.e. mouse
snap). User personalities and habits are not accounted for in any
of the GOMS models. All users are assumed to be exactly the same.
Except for KLM, the evaluators are required to have a fairly deep understanding
of the theoretical foundations GOMS, CCT or MHP. This limits the
effective use of GOMS to large entities with the finical power to hire
a dedicated HCI specialist or contract a consultant.
Conclusion
Aside from the weaknesses mentioned, GOMS is a useful tool for designing
and evaluating systems where human interaction is required. Once
descriptions of the steps the users will have to take are identified and
structured according to the Goals, Operators, Methods and Selection Rules,
a quantitative estimate of task execution time can be made. In some
cases, learning time can also be predicted. The four different variations
of GOMS address different situations, require different levels of understanding
by the evaluator and make different assumptions. Each has a place
is addressing interaction issues. The techniques can provide
a designer with information on how well users will eventually be able to
perform on a design and prevents the excuse that novice errors are to blame.
GOMS analysis is still an active research area. The work may eventually
lead to ‘canned’ user interface evaluation programs for programmers and
designers to use. These will be a tremendous aid for any programmer
because they would relieve them of the responsibility of judging the quality
of the interface he or she designs. Of course, the end users will
see the largest benefit.
References
1. Dix, A., Finlay, J., Abowd, G., Beale, R., Human
Computer Interaction , Prentice Hall, 1993
2. Lecture
Notes, Abowd, G., CS6751, Georgia Institute of Technology, Nov-1997
3. Kieras, D., John, B., The
GOMS Family of Analysis Techniques: Tools for Design and Evaluation,
CMU-HCII-94-106, 1994
4. Kieras, D., John, B., Using
GOMS for User Interface Design and Evaluation: Which Technique?,
June-1996
5. Kieras, D., John, B., The
GOMS Family of User Interface Analysis Techniques: Comparison and Contrast
6. Lecture Notes,
CS5724, Virginia Tech, Fall 1996