GIT-GVU-96-07

A Study of the Effect of Context and Test Method in Evaluating Safety Symbols

Jennifer Snow Wolff

The study measured the effect of context and test method in evaluating safety symbols. The study consisted of a 2x3 factorial test with context and no context on one axis and plausible and poor multiple choice distracters and open ended testing methods on the other 33 symbols were tested across all six conditions.

The study measured the effect of the quality of multiple choice distracters or alternative answers on scores. The open ended comprehension method was used as a control to measure the ability of multiple choice to capture subject responses. It was found that typical distracters obtained in an independent reputable, ANSI sponsored tests were below average in plausibility compared to distracters obtained through open-ended comprehension testing. Furthermore, it was shown that the low plausibility of those distracters, and the arbitrary limit in range of allowable answers led to inflated scores. The range of difference between low and high plausibility distracters was 30% across all 33 symbols.

Providing pictorial context in the test environment resulted in a more valid method of raising symbol scores. Context, in this study, was provided by 1 to 4 color photographs of probable environments where a symbol would appear. Context effects in testing were found for symbols low in external context cues in the symbols themselves. Context effects were not found in symbols which contained contextual detail. External context cues in the symbols were defined as environmental detail such as water, building structures and identifiable tools or machines. The average effect for low context symbols in the open-ended comprehension testing method was an increase of 15 percentage points. The average effect in the multiple choice method was an increase of 18 percentage points in the good distracter condition and 7 in the low plausibility distracter condition.

The ability of context to raise scores is significant because a valid method of testing which will also result in symbols which can exceed the ANSI 85% standard is important to producers of hazardous products for liability protection. The principle issue in products liability cases involving warning defects is whether the product failed to contain an adequate warning about the dangers inherent in using the product. (Grisim, 1993) Providing empirical proof that a comprehensible symbol was used is a critical feature of a symbol testing procedure.

The findings of the study are further significant, because it suggests that scores which placed symbols currently in the standard were invalid. Furthermore, it suggests that the inclusion of the multiple choice method, a commonly used method of symbol testing in laboratories across the country should be struck from the standard.

Testing methods, safety symbols, warnings, ANSI z535.3, standards, context, multiple-choice testing, iterative design, pictorials

PDF

Postscript