Supporting Learners as Users
College of Computing
Georgia Institute of Technologyguzdial@cc.gatech.edu
Software designers following a user-centered design philosophy try to provide software that helps users achieve their task goals. Word-processing programs help users to create, format, and spellcheck text, if that's the goal of the users. Checkbook programs help users to record transactions and reconcile their account with the bank, if that's their goal.
But sometimes, the user's goal is not the same as the explicit task that the software was designed for. That's the state that learners are in when they attempt to meet their learning goals through performing some task. For example, you may want to learn about physics by programming physics simulations, learn something about statistics by exploring in a spreadsheet or with Mathematica, or learn about yourself by keeping a personal journal in a word-processing program. Many adults in lifelong learning programs are attempting this kind of learning, where they are using their real work workstations to support their learning activity, too. The learning becomes even more complicated if the learner does not already know how to program or to use spreadsheets or Mathematica. This latter situation is common in undergraduate education where a student is asked to use the tool used by professionals while still learning the content of their domain (e.g., an engineering student learning to use a mathematical modeling language like Matlab while learning mathematical modeling or digital signal processing).
Learners don't have a single goal or task when they are using an application program, but two or more. The goals may be hierarchically arranged in terms of importance, but they are often pursued in parallel. Primarily, their task is to learn some content or maybe, to get a good grade, and to do whatever learning is necessary to get that good grade. Secondarily, their task may be to program, to build a spreadsheet, or to write. Related to that secondary goal may be to learn to program, to build a spreadsheet, or even to use the word processor. I call this model of learner as learner-as-user, which contrasts with:
Supporting learners-as-users is a complicated challenge. I have argued elsewhere that designing for learners is a wholly different activity than traditional user-centered design, which tends to focus on expert users . Learners are less motivated (or, perhaps just need more motivation to get through all the learning and doing tasks ahead of them), are changing in their knowledge and task characteristics more rapidly than experts, and know less about what they are doing. Even from a user-centered design perspective, most current applications and support (e.g., documentation, training, etc.) are not aimed to support the learner-as-user, since they rarely include support for the task of learning.
Consider instances of good support for users, such as the "Minimal Manual" approach . A minimal manual organizes documentation in terms of the tasks in which users will actually be engaging. Our current best methodology for designing minimal manuals is to use the GOMS model to identify the users' tasks and how they should be deconstructed in the manual . But GOMS is insufficient for modeling the activities in learnersGOMS assumes that the user knows their goals and operators, not that they will be learning them and having to choose between different goals in a hierarchy . Our current best practice is insufficient for meeting the needs of learners, and new methodologies and perhaps new kinds of support are needed to meet the needs of learners-as-users.
Learning is a conscious task, most cognitive science researchers now agree . Experience in project-based learning suggests that both learners do need support in understanding how to use existing applications to support their activities . Learners-as-users, with a huge hierarchy of goals, need enormous motivation to work through all of them, models of good process and good product to emulate, and scaffolding to insure that the enormity of lower-level goals does not prevent achieving the uppermost goals. Scaffolding is an education term used to describe the kind of support that a master craftsperson might offer to an apprentice in a traditional apprenticeship learning situation . A characteristic of most scaffolding is that it fades, or diminishes so as to allow the student to take over the role of the support. However, sometimes you want scaffolding to remain permanently, because fading the scaffolding would introduce a new and unnecessary learning goal . Scaffolding is one of the kinds of support that is needed to help the learner succeed as a user.
The question is how to provide the kind of support that a learner-as-user needs to succeed at the tree of goals facing her. In the rest of this paper, I present several instances of supporting learner-as-user. The first few are offered as relevant background, to show that learners can be successfully supported as users and to show how that has been achieved in the past. The latter two are instances from my own recent work at Georgia Tech's College of Computing in a class where students are learning object-oriented design and analysis, which are independent of any programming language, while they learn to program in a particular programming language, Smalltalk. In this more recent work, the students are being supported in an unmodified, commercial piece of software in a traditional classroom setting, and thus the work offers lessons on how to provide support to learners-as-users without modifying the software or the learners' situation.
Successful Support for Learner-as-User
Perhaps the first successful research study of learner-as-user was the thesis work of Idit Harel at MIT's Media Lab with Seymour Papert . Harel asked fourth graders to build educational software for third graders to help teach about fractions. In the fourth graders goal hierarchy were the goals of learning the programming language Logo (an unmodified, commercial version), learning to program at all, learning to design, learning what educational software was about, and, most importantly, learning fractions themselves. Harel found (through interviews and standardized tests) that her fourth graders really did not understand fractions when they began her project, nor did they know anything about programming.
By the end of the project, every student had developed a relatively complex piece of interactive software, and had learned a lot about Logo and fractions as well. In fact, the students who were part of Harel's project learned significantly more on all measures than students in control groups who had studied both Logo and fractions in more traditional settings. Harel's work was a great success for those who felt that learners could succeed as users. She showed that the hierarchy of goals could work in synergy to support better than expected learning for all of the goals. Harel's work was replicated and extended by Yasmin Kafai in several studies that explored very interesting variations , such as where the next group of fourth graders was aided by the (then) fifth graders as consultants.
Harel and Kafai supported their learners-as-users through a number of structures:
The work of Harel and Kafai has served as a model for supporting learners-as-users in other research projects (e.g., ), but it's an expensive model. It requires a great deal of labor and time. Cheaper and faster forms of support for learners-as-users have typically involved the use of specialized software. This "specialized software" is still software supporting a useful task other than learning (such as programming, data analysis, and visualization), but is especially designed to support learning, too. In contrast, most educational software aims at supporting learning (e.g., CAI, tutors, computer games), but does not also support the performance of a useful task other than learning.
The Boxer project at the University of California at Berkeley, led by Andrea diSessa, has successfully supported student learning in math , physics , biology , and other fields while the students learned to program in a specialized programming language called Boxer . Boxer is similar to the Logo programming language used by Harel and Kafai, but it offers special user interface additions which make it particularly well suited for learning programming and for using Boxer as an environment for working in, not just programming in. For example, all computation in Boxer is based on "boxes" which are general interface objects in which text, data, or even graphs can be stored and manipulated. There is little explicit support in Boxer for the elements in Harel's and Kafai's studies, so much of the labor and time intensive elements must still be provided in the classroom.
In my own dissertation work, I attempted to address some of the more expensive time elements. In Emile, I provided explicit and adaptable scaffolding to help high schools program physics simulations as a way to learn physics . Emile was similar to the Training Wheels approach , in that it prevented common learner error states by simply disallowing them. In Carroll et al.'s work, they determined the most common learner error states in a word processor, then created a new version of the word processor where those states were blocked. Students were able to learn more quickly and were able to transfer their learning to the full word processor.
In Emile, students began with a number of scaffolding "blocks" in place. For example, they could not type any programming code directly (thus preventing syntax errors). Instead, they could only compose code components from a library and customize them in pre-defined ways. Emile extended the Training Wheels approach in that a variety of kinds of blocks were available, and all were adaptable from within the program. When a students felt ready, she could turn off the block that prevented typing code in order to create code elements that could be combined with library elements. Emile was successful in that students using it were able to learn physics and programming, creating four physics simulations in three weeks.
Not all of the specialized software for learners-as-users is just to teach programming! For example, the Investigator's Workshop (IWshop), developed by Elliot Soloway and his Hi-CE Research Group at the University of Michigan, is aimed at supporting learners engaged in scientific inquiry. IWshop helps students to gather data, visualize their data, create models of their data, and compare and contrast their data and models. IWshop explicitly includes support for reflection (e.g., asking students their plans) as well as lots of guidance like that provided by good mediation. It is used in project-based learning classrooms that offer similar mediation and similar models of collaboration to that used in the Harel and Kafai studies .
While it is good to know that well-designed software can be built to support learners-as-users, most software is aimed at the more traditional computer user who only wishes to achieve their task goals. Thus, most software that a learner-as-user might want to use in combining a task with learning does not have any special features or special support infrastructure to facilitate the successful completion of a hierarchy of goals. The challenge, then, is figuring out how to support students in using traditional software for more-than-traditional activities. In the best case, the support should not require the learners to leave the situations in which they already find themselves, such as the workplace or the university classroom setting.
Supporting Use of Traditional Software in Traditional Settings for Non-Traditional Users
I teach a class at Georgia Tech, CS2390 Modeling and Design, in which students are expected to learn object-oriented analysis, design, and programming in a language independent manner while building simulations and user interfaces in a specific object-oriented programming language, Smalltalk. Smalltalk is an integrated language compiler and integrated development environment, and it's not an easy thing to learn (see for more on the challenge of learning Smalltalk). My students are in the position of being users of unmodified software like that used by professionals in their field, while they are still learning the knowledge of professionals in the field. As a researcher in educational technology, I have taken this opportunity to use my own class in exploring this problem.
The class is fairly traditional. It has been taught for ten-week terms for the last five years, with 75-100 students per class. Typically, there are three to four teaching assistants helping the professor with grading and addressing students' questions. Assignments are chosen to be interesting, but with such a large class, it's hard to believe that they are intrinsically motivating for all. Assignment examples are designing and building a simulation of a subway system, or a presentation device for a multimedia slide show.
In this section, I describe briefly two very different attempts to support learners-as-users in the context of the Modeling and Design class. Both attempts center on the mediation and collaboration aspects of the support that Harel and Kafai provided. In the first, students were provided with a static and carefully designed case base of successful projects developed by prior students in the class. In the second, students are, instead, provided with a flexible, collaborative space in which they can provide their own case base for each other.
A Case Base of Projects
STABLE is the SmallTalk Apprenticeship-Based Learning Environment . It is a case library of some 13 projects, most by students, that are presented across some 1200 web pages. Each of the projects in STABLE is presented in great detail (Figure 1):
Figure 1: Starting Page for a Project in STABLE
STABLE was provided to Modeling and Design students to use as they developed their own projects. Students were encouraged to use these as models of good projects (both process and product) in their assignments during the class, and to feel free to reuse any analyses or programming code that they found valuable in STABLE. The assignments were tuned to take advantage of STABLE, but were not significantly different from other assignments made in the same class . My hope was that STABLE would provide some of the elements that led to successful learners-as-users experiences:
Graduate research assistants and I conducted several studies of STABLE during its first use in Modeling and Design. Student performance on projects and exams were compared with prior offerings of the class on similar projects and similar exam questions. We found evidence that STABLE supported students in performance and in learning .
These two results are significant because they demonstrate support for learners-as-users, without additional resources or time during the class. Even the development cost for STABLE was relatively low since the majority of the cases were based on prior student work, which probably led to improved accessibility for the students than if the cases were drawn from industry or created especially for STABLE. The STABLE results show that a collection of accessible examples with connections to concepts does provide a large piece of the support that learners-as-users need.
However, the STABLE results were not all positive. In surveys and interviews, students kept telling us how much they disliked STABLE. They found it confusing, hard to navigate, and difficult to use. Nevertheless, the continued to use it a great deal. For example, the average length of a session with STABLE visited 30-40 pages.
Continued analysis of student usage patterns and interviews with students eventually led to the discovery that students use of STABLE was very different than what we expected. Where we provided a great deal of detail within a single project, we found that most students' visits involved multiple projects. What students found most valuable about STABLE was comparing and contrasting different projects. They wanted to see how the analysis process on these two projects compared, or how these two different user interfaces were implemented, or how these two different simulations were implemented. The next step is not obvious. Do we create static links between any two pieces of any projects that might be worth comparing? Do we develop a dynamic linking scheme that determines the best matches for the given point in the curriculum and the students' knowledge? Or, is the activity of trying to find two comparable components of projects an important activity that is leading to some of the performance and learning results?
A Collaborative Website that Contains Projects
In parallel with the work on STABLE, my colleagues and I were exploring collaborative learning environments to support project-based learning (e.g., ). At about the time we discovered the problem with STABLE's structure, we had developed a new kind of collarative space, called a CoWeb (for Collaborative Website, seehttp://pbl.cc.gatech.edu:8080/myswiki.1 ) . Based on the WikiWikiWeb by Ward Cunningham (http://c2.com), the CoWeb is a website with the simple rule that any page is editable by anyone and anyone can create new pages in the site. We decided to address our case library construction problem by asking students to build their own case library using the CoWeb.
Figure 2 shows a page in the Modeling and Design class CoWeb on the left, and on the right is the page that appears if you were to click on the "Edit this Page." The CoWeb allows the user to simply type text as if the page were an email note (e.g., a blank line between paragraphs gets translated into HTML paragraph markers). HTML can be used as the author wishes. Anything between asterisks is translated into a hypertext link. A phrase between asterisks, e.g., *Mark's Page*, creates a link to a page with that name, creating one if it doesn't already exist. A reference to a GIF or JPEG image address within asterisks is converted into an inline image. A search function and a list of all pages in the date-stamped order in which they were last edited make it easy to track new information and find old information.
Figure 2: A Page in the Class CoWeb (left) and its Edit View (right).
We created a CoWeb for the students in the Modeling and Design class (athttp://pbl.cc.gatech.edu/cs2390/1.html). We created a number of pages for different kinds of discussion and contributions, and created a few initial cases to begin populating a case library page (seen on the left of Figure 2). For example, we created a Smalltalk FAQ page and a page for questions and answers on each assignment. We created a Who's Who page where students were encouraged to create their own pages and describe themselves. We offered extra credit to students who would create cases from their best assignments. We also offered extra credit to students who would write essays about issues in the class, from design to programming to Smalltalk.
During the first few weeks, there was relatively little activity in the CoWeb. As the Midterm Exam approached, I posted a set of past midterm exam problems as a review, and created a page for each as a space for posting potential solutions and ask questions. The Midterm Exam Review served as an ice-breaker. It was an activity that was valuable to participate in, provided experience in writing in the CoWeb, but was not required. Soon after, all kinds of pages were being developed by the students, from cases (over 20 the first term) to essays on programming in Smalltalk. Non-traditional pages were also created. For example, one student created a page where he and other students could arrange for on-line gaming competitions.
We have continued to use the same CoWeb in the following offerings of the course. Students are still attempting the same kinds of problems as they were in the STABLE-based offerings of the course. The CoWeb case library is approaching 100 cases. The question-and-answer pages exist in a wide variety of styles. Non-traditional pages also continue to flourish, including an adventure game and even a posting of a song about the class. Valuable new essays and projects have appeared in the CoWeb, from a comparison of Java and Smalltalk, to discussions of design stylesall without my involvement.
We have not measured usage , performance, nor learning as carefully as STABLE yet. However, interviews and surveys suggest that the case library is frequently accessed and is probably as valuable to the students as STABLE was. We believe that the same kinds of mediation and collaboration support sare being provided. Further, the students are not complaining about the structure of the cases there.
What is most striking, however, is the increased motivation that we are seeing in the use of the CoWeb. We are finding that students who have graduated the class return to answer questions and to update their Who's Who pages or cases. Students continue discussions that were started terms ago, sometimes updating or correcting answers that were given in the past. Students spend a great deal of effort constructing pages of valuable information for future students.
The CoWeb, I believe, is providing many of the same supports as STABLE: Good examples linked to conceptual information. But in addition, the CoWeb is providing a collaboration space of which the students have taken active ownership, can contribute where they find most motivating, can find and lend support that best meshes with their goals, and is embedded within a huge support group of peer and more senior students. The CoWeb is motivating to use and update in ways that STABLE never was.
Modern cognitive science sees students not as vessels to be filled, but as active participants in their learning, where the learner is actively constructing knowledge often while constructing something else of value . Sometimes, learners use software to help them construct knowledge while constructing other things. In that case, learners are also users.
Students can be successful both as learners and as users who are trying to complete a task, as Harel and Kafai showed, but it can be expensive. We can reduce the costs by providing specialized software and by providing external supports for the learners-as-users. Below is the list, again, of what Harel and Kafai provided, and the support is being provided in other ways.
Supporting learners-as-users is an important challenge that requires new kinds of analysis, development, and implementation mechanisms. The research community is developing good models of what is necessary to support learners-as-users and how to provide this support at a reasonable cost.
An interesting question is whether the learner-as-user model is the exception or the rule. How often are experts learning new things, or trying to meet multiple learning-and-task goals at once, or finding themselves in situations where they need new expertise? Perhaps the expert-as-user model is the actual exception. If that is so, it may be that more of our support for users at all levels (e.g., professional development and training) needs to attend to features such as collaboration and motivation than it currently does.
This research has been funded by the EduTech Institute at Georgia Tech through a grant from the Woodruff Foundation, by the GE Foundation, and by the National Science Foundation (RED-9550458, CDA-9414227). Thanks to colleagues Amy Bruckman, Colleen Kehoe, Janet Kolodner, Mike McCracken, Wendy Newstetter, Jennifer Turns, and others for comments on this paper and more generally in support of this work.