In this paper, we report results from a survey that was posted on the Web for a month, in January of 1994. There were se veral goals motivating our survey. First, we wished to demonstrate a proof of concept for WWW technologies as a useful s urvey medium. Second, we wanted to beta-test the design and content of surveys dealing with the Web. Third, as mentioned , we hoped to begin to describe the range of Web users.
In one month, we had 4,777 respondents to our survey. Their responses helped us to begin to characterize WWW users, thei r reasons for using the WWW, and their opinions of WWW tools and technologies. However, as we will discuss, these respon ses are representative of a particular class of WWW users, and therefore cannot yet be generalized to the general WWW po pulation. For this reason, we plan to improve the survey and run it periodically in the upcoming years. This will allow us to track changes in the population as WWW evolves and matures. Furthermore, the survey results will help us extrapola te to the general Internet population.
Fortunately, the level of client interactivity supported by Hypertext Markup Language Plus [HTML+], i.e. forms, is suffi cient to enable low-overhead, point and click responses, as well as text entry. Further enhancing the attractiveness of using WWWW technologies is the Hypertext Transfer Protocol [HTTP] server's ability to retrieve documents (the actual sur veys) and execute programs (response logging software). Thus, not only are users able to respond to surveys in an intuitive, non-labor intensive manner, but the results can be logged automatically via the HTTP server.
The design of our survey attempted to exploit WWW technologies. Specifically, we created five separate surveys (with que stions on general background, HTML, HTTP, Mosaic, and WWW Browser/Internet Usage) and rudimentary logging software. The decision to create separate surveys was made for several reasons. First, we wanted users to be able to complete each sur vey in three to five minutes. If we had created one survey with all the questions, we felt that users might have been le ss likely to take the time necessary to complete the entire survey. Second, WWW browsers like Mosaic have difficulty man aging documents with large number of embedded forms. We intentionally designed the surveys to minimize the number of emb edded forms (but still received comments from users that display problems interfered with the overall aesthetics and eff ectiveness of the survey). Finally, categorizing questions facilitated clarity of intention. That is, the users knew bef ore-hand the area each survey was intended to cover and therefore did not end up midway through surveys finding themselv es unable to answer questions.
The logging software parsed the user's responses and append the results to file. Each entry in the logs was time coded, but machine names were not recorded. While this method ensured a moderate degree of animity (correlations between the ti me logged and the times recorded by the server log can still be derived) it primarily resulted from the use of National Center for Supecomputing Applications' (NCSA) httpd_1.0a5 server, which does not pass remote host information into the s ub-shell's environment.
As stated above, the surveys were divided into five categories: 1) general background questions, 2) HTML questions, 3) H TTP questions, 4) Mosaic questions, and 5) WWW Browser/Internet usage. We felt that this stratification was sufficient t o help us characterize WWW users, their reasons for using the WWW, and their opinion of WWW tools and technologies.
The surveys primarily used HTML+ forms. Specifically, radio button (buttons that only allow one answer to be selected pe r question) were used to list available choices or corresponded to numerical ratings. We intentionally designed as few q uestions as possible that required text entry as this imposes greater demands on the end-user and decreases control over of the content of the responses.
The survey was officially announced on January 17th, 1994, on the comp.infosystems.www newsgroup. The posting sim ply stated the impetus behind the survey (i.e., to provide the community with a characterization of WWW users) and the l ocation via the Uniform Resource Locator [URL] of the survey. Subsequent notices were mailed to NCSA's What New Page maintainers and the maintainers of the WWW server at CERN due to the high visibility of these servers. Several point ers from other documents on the Web were also observed.
The HTML Survey (12 questions) was intended to characterize how difficult HTML is for users to learn and use. Question a sked respondents about the number of HTML documents authored, whether or not they knew if HTML was an International Stan dards Organization (ISO) standard (HTML is not an ISO standard), how difficult/easy they found it to learn FORM and ISMA P, and if the documentation on HTML was current and easy to understand. We also asked background questions inquiring abo ut the number of years of programming experience, and the number of languages known. An additional question was added on Jan 21, 1994. It asked whether or not users had prior experience with Standard Generalized markup Language (SGML) (from which HTML is derived).
The HTTP Survey (14 questions) was primarily designed to determine the extent of knowledge people have about HTTP. Hence , respondents found questions asking about whether they knew about server scripts, modules, CGI scripts, and ISMAPs and if they had used any of these methods. We also asked if respondents operated a server (and if so, which one and what por t it listened to). Users were also asked to rate, if applicable, the existing servers at the time (CERN, Gn, NCSA, and P lexus) on a scale from one to nine.
The Mosaic Survey (5 questions) was intended to gain feedback from users of NCSA's Mosaic. This questionnaire was the sh ortest of all the surveys, (we had asked the staff at NCSA to submit additional questions). Basically, we wished to know what platforms users primarily used (this question was heavily biased - see Discussion below), what rating they would a ssign to Mosaic, how they perceived the support staff at NCSA, and how helpful they found the on-line help.
The WWW Browser/Internet Usage Survey (20 questions) sought to find out more about how and why people use their WWW brow ser. This category was broken down into five subcategories: frequency, motivation, typical activities, media type assess ment, and general usage traits. The first two categories are self-explanatory, while the others might require some expla nation. Since WWW browsers allow access to almost all Internet resources, we were interested in the degree to which thes e browsers are replacing the client software designed for each individual resource. Hence, we asked questions on browser use to access of gopher, archie, FTP, wais, etc., as well as questions on use for exploration and accessing other non-t raditional resources (e.g., weather). For the media type assessment, we wished to determine media preferences. The surve ys used radio and check box buttons for questions addressing motivation, frequency, and general usage traits, and used s calar radio buttons for the typical activities and media assessment questions.(1)
In the next section, we discuss the findings from each survey, followed by a discussion of these results.
The results indicate that 56% of respondents are between the ages of 21 and 30, 94% are male, and 69% are located in Nor th America. Over 97% percent report using Mosaic as their browsing tool, 88% percent use UNIX as their platform, and 55% use single-user machines (though, as we discuss in the Shortcomings section, the latter results must be interpreted wit h care) (see Table 1).
In terms of occupation, 45% of the respondents describe themselves as professionals, and 22% as graduate students (the t wo largest categories). Thirty-six percent of the respondents claim to spend over 10 hours/week exploring the Internet, while 42% claim to spend 0 to 5 hours/week. Of interest to enterprises contemplating commercial use of the Internet, 55% of the respondents say they might be willing to pay fees for access to WWW information repositories, while 40% say they would not (see Table 1).
We surveyed users as to how often they used their WWW browser, instead of accessing specific client services (e.g., Wais , FTP, etc.), where 1 = "never" and 9 = "always." The results indicate that, overall, users show a s trong preference for using their WWW browser for general Internet exploration and instead of the standard Gopher and Wai s clients. They show a slight preference for using a Web browser for text retrieval and Archie services. They are neutra l in their preference for using WWW browsers to access publisher information, weather, and news of research activities a nd abstracts (see Table 8).
Finally, we asked users to rate their preferences of the various media supported by the Web. Users show a strong prefere nce for keyword search and embedded images. They show a preference for text and spawned images, and a slight preference for movies and sounds (see Table 8).
The analyses of the HTML survey offer less insight. Here, we note that the number of years programming interacts strongl y with the number of programming languages known (G2(16) = 733.633, p < .001; C(16) = .514). This follows intuition, as users with few years programming know few languages and users with over twelve year s programming correspond to knowledge of over twelve languages known. A moderate interaction was observed between the nu mber of HTML documents authored and uncertainty of whether HTML is an ISO standard (G2(8) = 6 1.080, p < .001; C(8) = .210).
Numerous interactions exist between the questions in the HTTP survey, though, most are as expected (see Table 14). Of no te, we found a strong interaction between knowledge of ISMAPs and ISMAP use (G2(2) = 277.407, p < .001; C(2) = .774) along with knowledge of CGI scripts and CGI script use (G2(2) = 155.007, p < .001; C(2) = .582). As expected then, knowledge of server scripts relates to knowledge and use of CGI scripts and ISMAPs , with the latter having a slightly weaker association. Similarly, we observe that operating a server interacts moderate ly with knowledge and use of CGI scripts and ISMAPs.
Analysis of the Mosaic survey (see Table 13) revealed that those users who used the Help menu on a weekly basis were mor e inclined to view the on-line help on Mosaic as useful (G2(6) = 124.135, p < .001; C(6) = .244).
In terms of the Usage Survey (see Table 14), the most frequent Web users appeared to explore the Internet the most (G 2(10) = 312.501, p < .001; C(10) = .370). Although a mild interaction, we note that as the preferenc e for using Web browsers for Internet exploration increased, users appeared to prefer text-based search; as preference f or using Web browsers for Internet exploration decreased, users preferred keyword search (G2( 4) = 312.501, p <.001; C(4) = .103).
We also encountered difficulties with keeping track of users across surveys. This problem results mostly from the statel ess nature of the HTTP protocol. That is, the server does not keep track of what machine accessed which documents; this is only logged to file. This meant that we were unable to take the answers submitted by a user for the General Survey an d perform analyses against questions in the other surveys. Hence, we could not test for possible interactions like the a ge of the users and user media orientation. There are several ways which we can avoid this issue in the future. One meth od would be to have each user provide a unique key upon starting the first survey, and then use this key for subsequent submissions. Hence, all responses to the surveys could be logged with a unique key for each user. This would enable the cross-question analyses we desire. This method adds additional burden to users by making them responsible for rememberin g the key for future surveys (which may be completed several days later). Another method would extensively utilize the a bility of HTTP servers to execute programs. Under this scheme, a program would produce the desired survey and attach a h idden key with the survey. Upon receiving the completed survey, the program would create another survey and, again, enco de a key within the document. As with the previous method, cross-survey question analysis is enabled, but this time, we impose a potentially severe time constraint on the user, i.e., the time to complete all the surveys. If a user decides t o stop participating in the currently returned survey, any future submissions will require complete re-entry of all surv eys, which is undesirable. Finally, HTTP could be modified to make it into a state machine, though the likelihood of thi s happening is uncertain.
Similar server related problems occur if we accept only fully completed surveys. While we gain from having consistent nu mber of responses to each question in each survey, we would have to develop software that checked each answer, and creat ed an additional document containing the questions that were not answered. The software would also have to keep track of the previous answers and, once the other answers were successfully returned, merge the results into one entry. We will most likely implement this method in future surveys.
In addition, at the time of our survey, few browsers and platforms supported the HTML+ forms extension. Thus, our result s are biased in favor of those that did support Forms (i.e., XMosaic for UNIX). Presently, more platforms and browsers s upport the extension, thus we expect to eliminate the bias with subsequent survey runs.
A final bias is due to a sampling problem, that is, the self-selected nature of survey respondents. Clearly, frequent We b users were more likely to answer our survey, and hence skew the results. We hope that future deployments of the survey will begin to attract responses from less-frequent users, and thereby allow us to characterize the changing Web user po pulation.
Given the dynamic nature of WWW use and technologies, we believe that surveys run twice a year ought to provide an optim al trade-off between maintaining respondents from survey to survey and charting the Web's growth and changes. In additio n, we hope that the WWW community will allow us to remain the sole Web surveyors in this domain. We fear that if other r esearchers clutter the Web with similar surveys, the overall utility of such surveys will be greatly diminished. In ligh t of such a request to the community, we gladly open ourself to suggestions and specific research agendas of other resea rchers.
Berners-Lee, T. (1994) Hypertext Markup Language Plus. URL:http://info.cern.ch/hypertext/WWW/MarkUp/HTMLPlus/htmlplus_1.html
Berners-Lee. T. (1993) Hypertext Transfer Protocol. Internet Engineering Task Fo rce Working Draft, 5 Nov 1993. URL:ftp://ds.internic.net/internet-drafts/draft-ietf-iiir-http-00.txt
Berners-Lee, T. (1994) Uniform Resource Locators. Internet Engineering Task Forc e Working Draft, 21 March 1994. URL:ftp://ds.internic.net/internet-drafts/draft-ietf-uri-url-03.txt
Ragget, D. (1994) Hypertext Markup Language Plus. URL:ftp://ds.internic.net/internet-drafts/draft-raggett-www-html-00.txt
MIMI RECKER received her Ph.D. from the University of California, Berkeley, in 1992. She is currently a Research Scienti st in the College of Computing at the Georgia Institute of Technology. Her research interests include cognitive science approaches to learning, metacognition, instruction, interactive learning environments, human-computer interaction, cogni tive modelling and multimedia.
Table 1: Results from the General Survey -------------------------------------------------------------------------------------------- Question 1. Mac PC Unix VMS Other Total Primary Plat 48 / 66 / 1181 / 33 / 7 / 1335 form / % 3.60 4.94 88.46 2.47 0.01 Question 2. Cello Lynx Mosaic Other Samba Total Primary 1 / 26 / 1294 / 6 / 4 / 1331 Browser / % 0.08 1.95 97.22 0.38 0.30 Question 3. No Yes Total Shared 596 / 736 / 1332 Machine/ % 44.74 55.26 Question 4. Under 5 6 to 10 11 to 20 21 to 30 31 to 40 41 to 50 50 + Total Hrs in Front of 2 40 102 245 295 273 360 1344 Computer / % 0.15 7.59 7.59 18.23 21.95 2.98 26.79 Question 6. Asia Australia Europe North South Total Amer. Amer. Location / % 10 / 27 / 377 / 910 / 2 / 1326 0.75 2.04 28.43 68.63 0.15 Question 7. UGrad. Admini- Business Faculty Grad. Profess- Other Total Student stration Student ional Occupation / % 188 38 / 53 / 94 / 287 / 594 / 75 / 1329 14.15 2.86 3.99 7.07 21.60 44.70 5.64 Question 8. Maybe No Yes Total Willing to Pay 727 532 66 1325 Fees / % 54.87 40.15 4.98 Question 9. Under 20 21 - 25 26 - 30 31 - 35 36 - 40 41 - 50 50 + Total Age / % 85 / 380 / 362 / 218 / 144 / 103 / 26 / 1344 6.32 28.27 26.93 16.22 10.71 7.66 1.93 Question 10. Female Male Total Gender/ % 67 / 1247 / 1314 5.10 94.90 --------------------------------------------------------------------------------------------
Table 2: Results from the HTML Survey ------------------------------------------------------------------ Question 1. None 1 to 5 6 to 10 11 to 20 Over 20 Total Num Docs 52 / 189 / 102 / 86 / 270 / 701 Authored / % 7.42 26.96 14.55 12.27 38.52 Question 2. Yes No Unsure Total Is HTML ISO Stan 21 / 211 / 463 / 701 dard / % 3.00 30.10 66.05 Question 3. None 1 to 3 4 to 6 7 to 12 Over 12 Total Years of Program 21 / 55 / 145 / 233 / 240 / 701 ming / % 3.00 7.85 20.68 33.24 34.24 Question 3. None 1 to 3 4 to 6 7 to 12 Over 12 Total Num Programming 16 / 111 / 292 / 172 / 106 / 701 Langs / % 2.28 15.83 41.65 24.11 15.12 Question 12. Yes No Total Prior SGML Use / 31 / 333 / 364 % 8.52 91.48 ------------------------------------------------------------------
Table 3: Results from the HTML Survey --------------------------------------------------------------------- Question Mean Std. Dev. Total Question 5. < 1 - 9 > 2.258 1.377 654 Hours taken to learn HTML Question 6. < easy (1) - hard (9) > 2.132 1.393 651 Overall, learning of HTML Question 7. < easy (1) - hard (9) > 3.458 1.749 295 Learning Forms Question 8. < easy (1) - hard (9) > 3.781 2.072 201 Learning ISMAP Question 10. < easy (1) - hard (9) > 3.951 2.360 551 Finding up to the minute HTML documentation Question 11. < easy (1) - hard (9) > 3.064 1.681 626 Understanding of HTML documentation ---------------------------------------------------------------------
Table 4: Results from the HTTP Survey --------------------------------------------------------------------- Question 1. Yes No Total Operate HTTP Server / % 378 / 103 / 481 78.59 21.41 Question 2. GN or Cern NSCA Plexus Other Total If Yes, Which Server / % 28 / 271 / 48 / 30 / 377 7.43 71.88 12.73 7.96 Question 3. 80 8000 8001 8080 Total Server Port Number / % 279 / 10 / 13 / 16 / 354 78.81 2.82 3.67 4.52 Question 4. Yes No Total If no, Able to Add to DB / % 122 / 32 / 154 79.22 20.78 Question 9. Yes No Some Total Knowledge of Script / % 260 / 66 / 130 / 476 58.82 13.87 27.31 Question 10. Yes No Some Total Knowledge of Modules / % 73 / 341 / 57 / 471 15.50 72.40 12.10 Question 11. Yes No Some Total Knowledge of CGI / % 190 / 191 / 87 / 471 40.60 40.81 18.59 Question 12. Yes No Total Written CGI scripts / % 177 / 285 / 462 38.31 61.69 Question 13. Yes No Some Total Knowledge of ISMAP/ % 138 / 217 / 111 / 466 29.61 46.57 23.82 Question 14. Yes No Total Used ISMAP scripts / % 128 / 344 / 472 27.12 72.88 ---------------------------------------------------------------------
Table 5: Results from the HTTP Survey ------------------------------------------------------------------------- Question Mean Std. Dev. Total Question 5. < horrid (1) - excellent (9) > 5.815 1.854 292 Overall Rating of server Question 6. < horrid (1) - excellent (9) > 6.041 1.958 98 Overall Rating of GN's server Question 7. < horrid (1) - excellent (9) > 7.453 1.239 750 Overall Rating of NCSA's server Question 8. < horrid (1) - excellent (9) > 6.788 1.686 170 Overall Rating of Plexus's server -------------------------------------------------------------------------
Table 6: Results from the Mosaic Survey ----------------------------------------------------------- Question 1. UNIX PC Mac Total Which Mosaic Plat 1006 / 48 / 25 / 1079 form / % 93.23 4.45 2.32 Question 4. Daily Weekly Monthly Never Total Freq of Use of Help 89 / 306 / 326 / 356 / 1077 Menu / % 8.26 28.41 30.27 33.05 Question 5. Yes No Some Total Is On-line Help 551 / 74 / 428 / 1053 Useful / % 52.33 7.03 40.65 -----------------------------------------------------------
Table 7: Results from the Mosaic Survey ------------------------------------------------------------------------- Question Mean Std. Dev. Total Question 2. < horrid (1) - excellent (9) > 8.086 0.881 1081 Overall Rating of Mosaic Question 3. < horrid (1) - excellent (9) > 7.458 1.520 334 Rating of Support Staff @ NCSA -------------------------------------------------------------------------
Table 8: Results from the Usage Survey ------------------------------------------------------------------------------ Question Mean Std. Dev. Total Question 3. < never (1) - always (9) > 5.81 3.39 1145 Use Browser to Retrieve Documents Question 4. < never (1) - always (9) > 7.42 3.96 1048 Use Browser Instead of Gopher Question 5. < never (1) - always (9) > 7.60 5.44 1007 Use Browser Instead of Wais Question 6. < never (1) - always (9) > 5.65 8.17 1026 Use Browser Instead of Archie Question 7. < never (1) - always (9) > 4.20 5.15 1116 Use Browser to View Research Question 8. < never (1) - always (9) > 7.30 2.23 1138 Use Browser to Explore Internet Question 9. < never (1) - always (9) > 4.10 6.32 1139 Use Browser to Retrieve Weather Information Question 10. < never (1) - always (9) > 4.97 4.41 1127 Use Browser to Access Reference Materials Question 11. < never (1) - always (9) > 4.16 4.72 1116 Use Browser to Access Publishers, Periodicals, etc. Question 12. < never (1) - always (9) > 4.33 5.13 1122 Use Browser to Access Call for Papers, Proposals, etc. Question 13. < love (1) - hate (9) > 6.62 2.57 1140 User `s Preference of Text Question 14. < love (1) - hate (9) > 7.30 2.31 1140 User `s Preference of Embedded Images Question 15. < love (1) - hate (9) > 6.48 3.42 1124 User `s Preference of Spawned Images Question 16. < love (1) - hate (9) > 5.41 4.33 1089 User `s Preference of Sounds Question 17. < love (1) - hate (9) > 5.83 4.19 1097 User `s Preference of Movies Question 18. < love (1) - hate (9) > 7.93 1.87 1130 User `s Preference of Search by Keyword ------------------------------------------------------------------------------
Table 9: Results from the Usage Survey ------------------------------------------------------------------------------------- Question 1. Over 9/ 5 to 8/ 1 to 4 / Few times Once / Once/ Total day day day / week week month Frequency of Browser 235 202 481 199 23 9 1153 Use / % 20.38 17.52 41.72 17.26 1.99 0.78 Question 19. Search Text Visual Total Search Orientation / 319 432 372 1344 % Question 20. Under 5 6 to 10 Over 10 Total Hours Exploring the 484 407 252 1153 Internet / % 41.98 35.30 21.86 -------------------------------------------------------------------------------------
Table 10: Results of Chi-Square for General Survey ---------------------------------------------------------------------------------------------- Interaction Chi-Square Value Degrees of Probability Cramer V Freedom Machine Shared by Hrs in Front of Cptr 43.834 6 0.000 0.183 * Question 3 by 4 Machine Shared by Domain 139.779 6 0.000 0.324 Question 3 by 5 Machine Shared by Location 24.233 4 0.000 0.135 * Question 3 by 6 Machine Shared by Occupation 199.290 6 0.000 0.388 Question 3 by 7 Machine Shared by Willing to Pay Fees 30.632 2 0.000 0.152 Question 3 by 8 Machine Shared by Age 122.729 6 0.000 0.306 * Question 3 by 9 Hrs in Front of Computer by Occupation 169.460 36 0.000 0.147 ** Question 4 by 8 Hours in Front of Computer by Fees 23.295 12 0.000 0.094 ** Question 4 by 8 Hours in Front of Computer by Age 90.139 30 0.000 0.118 Question 4 by 9 Occupation by Fees 52.816 12 0.000 0.141 Question 7 by 8 Occupation by Age 835.564 36 0.000 0.326 ** Question 7 by 9 Willing to Pay Fees by Age 26.628 12 0.009 0.101 * Question 8 by 9 ----------------------------------------------------------------------------------------------NOTES: Since there were only two responses for the under 15 category in Question 9 (Age), the responses were not include d into the Chi-Square calculations.
* Denotes a negative Kendall Tau-B result
** WARNING: More than one-fifth of the cells were sparse (Frequency < 5) significance suspect
Table 11: Results of Chi-Square for HTML Survey ------------------------------------------------------------------------------------------- Interaction Chi-Square Value Degrees of Probability Cramer V Freedom Documents Authored by ISO Standard 61.080 8 0.000 0.210 Question 1 by 2 Documents Authored by Years Program. 30.766 16 0.014 0.105 * Question 1 by 3 Documents Authored by Languages 30.766 16 0.014 0.105 * Question 1 by 4 Standard by Years Programming 19.541 8 0.012 0.119 Question 2 by 3 Standard by Languages 19.722 8 0.011 0.119 Question 2 by 4 Years Programming by Languages 733.663 16 0.000 0.514 Question 3 by 4 -------------------------------------------------------------------------------------------* Denotes a negative Kendall Tau-B result
Table 12: Results of Chi-Square for HTTP Survey -------------------------------------------------------------------------------------------- Interaction Chi-Square Value Degrees of Probability Cramer V Freedom Operate by Access to WWW Database 27.185 1 0.000 0.388 Question 1 by 4 Operate by Knowledge of Server Script 69.383 2 0.000 0.383 Question 1 by 9 Operate by Knowledge of Module 7.643 2 0.22 0.128 Question 1 by 10 Operate by Knowledge of CGI Script 62.132 2 0.000 0.365 Question 1 by 11 Operate by CGI Script Use 50.759 1 0.000 0.316 Question 1 by 12 Operate by Knowledge of ISMAP 44.283 2 0.000 0.309 Question 1 by 13 Operate by ISMAP Use 24.838 1 0.000 0.224 Question 1 by 14 Server by Knowledge of Server Script 41.596 6 0.000 0.236 Question 2 by 9 Server by Knowledge of Module 145.472 6 0.000 0.444 Question 2 by 10 Server by Knowledge of CGI Script 31.442 6 0.000 0.206 Question 2 by 11 Server by CGI Script Use 23.121 3 0.000 0.252 Question 2 by 12 Server by Knowledge of ISMAP 10.919 6 0.091 not significant Question 2 by 13 Server by ISMAP Use 8.859 3 0.031 0.155 Question 2 by 14 Server Script by Knowledge of Module 53.810 4 0.000 0.239 Question 9 by 10 Server Script by CGI Script 150.649 4 0.000 0.401 Question 9 by 11 Server Script by CGI Script Use 112.718 2 0.000 0.494 Question 9 by 12 Server Script by ISMAP 97.941 4 0.000 0.325 Question 9 by 13 Server Script by ISMAP Use 54.503 2 0.000 0.340 Question 9 by 14 Know. of Module by CGI Script 47.495 4 0.000 0.226 Question 10 by 11 Know. of Module by CGI Script Use 31.997 2 0.000 0.264 Question 10 by 12 Know. of Module by ISMAP 46.291 4 0.000 0.224 Question 10 by 13 Know. of Module by ISMAP Use 24.985 2 0.000 0.231 Question 10 by 14 Know. of CGI by CGI Script Use 155.007 2 0.000 0.582 Question 11 by 12 Know. of CGI by ISMAP 72.535 4 0.000 0.281 Question 11 by 13 Know. of CGI by ISMAP Use 40.937 2 0.000 0.296 Question 11 by 14 CGI Use by ISMAP 71.809 2 0.000 0.398 Question 12 by 13 CGI Use by ISMAP Use 63.903 2 0.000 0.350 Question 12 by 14 ISMAP by ISMAP Use 277.407 2 0.000 0.774 Question 13 by 14 --------------------------------------------------------------------------------------------
Table 13: Results of Chi-Square for Mosaic Survey ------------------------------------------------------------------------------------------ Interaction Chi-Square Value Degrees of Probability Cramer V Freedom Use of Help by On-Line Docs Helpful 124.135 6 0.000 0.244 Question 4 by 5 ------------------------------------------------------------------------------------------
Table 14: Results of Chi-Square for Usage Survey ------------------------------------------------------------------------------------------ Interaction Chi-Square Value Degrees of Probability Cramer V Freedom Frequency of Use by Orientation 21.060 10 0.031 0.097 Question 1 by 19 Frequency of Use by Hours Exploring 312.501 10 0.000 0.370 Question 1 by 20 Orientation by Hours Exploring 23.518 4 0.000 0.103 Question 19 by 20 ------------------------------------------------------------------------------------------