Designing for Learners

Group Leaders: Elliot Soloway and Mark Guzdial

Participants

1. Vision

Designing technology for learners is a completely different activity than designing for professionals or experts. Learners do not necessarily know what they want to do with technology, or how to do it, or even if they want to do it. For a learner to be successful with the technology, the interaction must be supportive and empowering. For the learner to actually learn with the technology, the interaction must also encourage reflection and motivation.

These are high standards for a piece of software or a computerized device. What makes these standards so challenging is that we have very little knowledge in the design community on how to do achieve them. There are four key research questions important to the pursuit of design for learners?

In this section of the report, we begin by describing some design processes and considerations developed by teams in the working group on Designing for Learners. With those designs as concrete examples, we go on to address each of the four key questions.

2. Design Examples

To make the issue of designing for learners more concrete, the working group on Designing for Learners spent time in small groups doing design for learners. No designs were completed in the short time available, so no evaluation was attempted. However, the designs served as an example of how design for learners must address the first three questions: Three of those design discussions are summarized here: Design of a physical learning space, design of an on-line museum exhibit, and design of a programming language for learners.

A. Design of a Physical Learning Space

Characterizing learners and their activities: The group members spent little time on the questions of characterizing the learners and their activities in a physical learning space. Instead, they were more interested in the broader question of what should replace traditional physical classrooms given advanced technologies, across different kinds of learners and learning activities.

Genres of technology: Instead, the group members began by identifying metaphors that served as examples of comfortable and successful learning environments but challenged traditional models of learning spaces. They talked about sports arenas, theaters, informal banjo lessons, and even sitting on a parent's or grandparent's knee. The goals were to enable participants in the learning process to see one another, to be intimate, and to be comfortable with the space. At the same time, the participants wanted to develop metaphors that would allow integration of hardware and software into the physical space.

The metaphor that the group found most valuable was "The 24 Hour Classroom." Educators in general, and university educators in particular, have been critized as not being productive enough. The theme of a 24 Hour Classroom emphasizes the constant availability of learning opportunities made available through technology. The teacher can be available whenever she or he has the opportunity to visit synchronous or asynchronous collaborative spaces, while other educational opportunities (e.g., fellow students, multimedia resources, diverse learning aids and guides) might be made available whenever the student has the opportunity to engage them. The 24 Hour Classroom is particularly attractive when considering the needs of adults engaged in lifelong learning.

Architectures and implementation strategies: The greatest research challenge in this metaphor is to explore mechanisms for realizing an integration of the real (physical) and virtual classrooms. We expect that students will continue to gather in classrooms and lecture halls, and that adult learners will do their work and learning in physical spaces. Integrating physical and virtual spaces could involve a number of different activities, combined in many different ways, such as:

B. Design of an On-Line Museum Exhibit

Characterizing learners and their activities: The designers noted characteristics of the kinds of activities desired for the learners in the exhibit: Genres of technology: The designers identified goals that they had for their design which determined the forms that they desired: Architectures and implementation strategies: There are a variety of research issues to be addressed before museum exhibits could be built that enable the desired learner activities and meet the design goals:

C. Design of a Programming Language for Learners

Characterizing learners and their activities: Programming as an activity for learners has a long history, from efforts to have students construct interesting artifacts by programming (e.g., Boxer, Emile), to using programming as a medium for learning higher-order thinking skills (e.g., Logo has been used in this light). The participants in this subgroup began by asking why they might want learners to use a programming language at all. Genres of technology: Next, the participants asked themselves what was wrong with current programming languages. They used Logo as an example language to critique. They came to the realization that the syntax of the language is not the most critical aspect of a language. In Logo's case, in particular, the "turtle" as a computational object is far more important to Logo than the syntax. In general, the problems with current programming languages for learners are more cultural and social than computational. A critical goal for a programming language for learners is that it have an epistemology ­ a theme, or set of embedded ideas, for how all the pieces of the language fit together and are used in solving computational problems. The participants identified several important ideas related to the value of an epistemology in a programming language: Finally, the participants returned to the issue of programming language for learners. Why should programming languages be used in education?

3. Research Areas and Issues

3.1. How do we characterize learners and their activities?

Developing an in-depth understanding of learners' unique needs and interests at the cognitive, organizational, and socio-cultural levels is a first step in designing meaningful and productive technologies for learning. We need to determine the goals that students need to pursue in learning. The need is both to take advantage of students' interests and intrinsic motivation, and also to inculcate goals that meet curricula and make those goals engaging. In either case, we need to develop models of how these learning goals can promote deep and rich learning. The learning goals have to be defined in conjunction with goals of the corresponding content areas using standards established by the National Council of Teacher in Mathematics (NCTM) and other educational standards groups as exemplary guidelines. The development of a design model that places the learner at the center and situates the learning process within the larger educational context is key to this process and will result in the identification of design features (and under which conditions) that will make significant contributions to students' learning.

CHARACTERIZING USERS AS LEARNERS

Current models of human-computer interaction place the user at the center of the software design process by defining the tasks that need to be undertaken by the software, the tools that are provided to cope with the task, and the interfaces to those tools. Learners introduce a different kind of user with distinctive features: whereas professionals know the domain, are motivated, and are an homogeneous population with the goal to increase their success, students do not know the domain, often are not motivated, and encompass very diverse populations. Models of learner-centered design need to take into account these features and move beyond "ease of use" to a model in which the focus is on "ease of adaptability". Furthermore, the research on gender differences points out that learners may have different interests that lead them to begin and continue their interaction with technology.

This focus becomes particularly relevant in the case of young learners. Technologies for learning are to serve the needs of learners of all ages in various content areas. There is little known, if anything, about interface design for young children as learners. What interface design and features help young children interact with a piece of software and facilitate their learning process? What structures and representations of software do facilitate the entrance of learners into a content area?

A further aspect to be considered in learner-centered design is the perspective of life-long learning. In its current use, the notion of learners is limited to school populations. Yet we need to recognize that learning is not just for students in the classroom but professionals are (or should be) constantly learning too. Moreover, when the professional is acting as a learner, that person is susceptible to all the challenges faced by students. How does one design systems for life-long learning which can be of use to a learner over a lifetime?

MEETING THE NEEDS OF THE BROADER AUDIENCE

Learners are not only found in formal educational settings such as classrooms and schools but also in informal learning places such as museums and afterschool organizations. What are the needs of learners in different learning places, how are these needs distinctive, and where do they overlap? Learner-centered design needs to be concerned also with the larger organizational setting as the integration of technology within the pedagogical and administrative nature of the learning place is critical.

A key partner in the educational process is the teacher. The potential roles of teachers within the computer-based learning environment require further investigation. How can the use of technology facilitate and support teachers' interactions with and coaching of students in the learning process? Within this context, we also need to address the needs of teachers as learners. Which features of technology create a learning environment that facilitates teachers' development of pedagogical knowledge and content knowledge. What are models of learner-centered design that see the conjunct development of software for students and teachers?

DEVELOPING A NEW DESIGN PROCESS

Developing models of human computer interaction that place learners at the center require a new design process that realizes that learners are moving target: as they are interacting with a piece of software, they are learning, and so does their interaction change over time. What is the design cycle when the task is to characterize users in terms of distributions and rates of change rather than static descriptions? What models of the design cycle take into account that software is being used by learners at different stages? How do we evaluate software that fits a dynamic model of learner development?

One goal should be to develop operational models of software design that delineate the stepping stones in the iterative design process. In conjunction with this approach, we need process models of students' thinking and learning that can provide guidance in defining land marks in software achievement. Ultimately, we cannot consider the software design process separately from the students' learning process as students' interactions with interface features change. Crucial might be here to identify those features of systems which should stay stable and those that can fade or disappear over time. The fading away gives learners more degrees of freedoms and addresses the critical issue of diversity in learners thinking and approaches. In this process, we need to define features of software modules that facilitate connections to other pieces of software‹that allow the learner to design products or temporary states of it and export it to other software.

3.2. What are the genres of technology to be designed and how do we design them?

There are a wide variety of potential genres for interactive computational media. It's useful to think of the potential genres as having both functional roles and supportive roles. The functional roles enable students to undertake activities that create opportunities for learning: Creating a Web site, designing a video game, writing a paper or a book. The supportive roles come in two kinds: (a) those that enable the student to succeed at the function where they might not otherwise and (b) those that facilitate learning. We recognize that experience, no matter how compelling, is not the same as learning ­ activity alone does not lead to learning without reflection. The supportive role must be there and must be deeply intertwined with the functional role.

FUNCTIONAL ROLES

The functional roles can build upon a student's interests or intrinsic motivations to create an opportunity for learning. There are a great many activities which a student may find exciting and intriguing which might not be possible or accessible without a computer. The challenge is providing advanced functionality at a level that students find approachable. We might meet this challenge by providing appropriate representations, metaphor-based interfaces, and encouraging interconnections between students' existing knowledge and challenges in a new setting.

We can define some of the functional roles that deserve exploration in a research program because of their common appearance in a variety of genres of interactive computational media. For example, our design example on programming languages for learners points to several research issues that pertain to functional roles:

For students to learn through other tasks (such as scientific computing and visualizations), there are other research issues to be explored, such as:

SUPPORTIVE ROLES

All users need functionality. In a research program to support learners, the crux is the support. Learners need special services to facilitate their success and their learning in a genre of computational media. We can identify several kinds of support which genres of computational media may provide, such as, cognitive apprenticeship, goal-based scenarios, and others. What is particularly important for the research community is that a single supportive role (e.g., collaboration) may be combined with a number of functional roles (e.g., writing, model-building) to create several genres of computational media for learners (e.g., collaborative writing environments, interactive model-building for workgroups, etc.). Thus, research in a supportive roles may broadly support a variety of current and future genres of computational media for learners.

One type of supportive role is to enable and facilitate collaboration as a mechanism both for facilitating success (e.g., groups can often do things that individuals cannot) and facilitating learning (e.g., learning can be facilitated by articulation, such as explaining to others what you are doing). Much work and learning is accomplished in the context of social activity. New software systems offer access not only to extensive development of educational material, but to varied and far-flung interactions between people. Users will have the ability to connect with friends, colleagues, and experts who previously would have been inaccessible, or at best by phone or mail. Research is needed on how users will interact with each other, how they will socialize in the new network communities, how different access protocols will exist across communities and how users will switch. Some of the research issues involved in the use of collaboration as part of the supportive role of a genre:

Another important supportive role is the access and utilization of diverse sources and resource materials across wide area networks. Research issues in making these sources broadly useful as supportive roles include:

A broad supportive role is scaffolding. Scaffolding is an education term for the kind of support provided to facilitate student success at an activity. Typically, it involves modeling a process (sometimes simplified at first) for a learner, coaching the learner through that activity, and providing opportunities for the student to articulate what has been learned (by teaching others, by writing instructions, by responding to prompts, etc.) Key in the concept of scaffolding is that the scaffolding can be faded, i.e., reduced so that the student can learn to succeed without the support. We are only just learning how to provide scaffolding in software, and there are a wide variety of supportive role research problems surrounding the issue of scaffolding:

Learning environments might require a model of the user and the domain in order to offer customized support to the student. Such support can be open-ended, non-intrusive, and can reside within authentic and community learning situations. They can be built within a community of practice, such as the medical field, and can actively engage the learner. Some may use planning or plan recognition research strategies to recognize how learning is phased and to guide it. Advanced model-based systems can involve parameters and variables which the user can change so that he or she can get his hands on the model and can change the internal model. Technological breakthroughs allow us to dissect the domain and control knowledge, to represent the user's knowledge and goals, and to represent and structure pedagogical knowledge. Research issues here fall into three categories: modeling the domain, the student model, and tutoring strategies.

Modeling the domain
Modeling the student
Modeling tutoring strategies

3.3 What are the architectures and implementation strategies for technology designed for learners?

A great limitation to the wide-scale use of new genre of interactive computational media is the inherent difficulty in creating educational software. The issue of implementation is important to the designer because the available tools for realization of a design both constrains the design process and highlights possibilities. Tools for learning environments (as opposed to simply tools for authoring educational software) are analogous to current educational authoring tools except that they must address the unique needs of learners at several levels, including the organizational and social cultural levels. They must fit into the educational system, be applicable to designing for learners, meet the needs of teachers as learners, and must address the diversity of genres of software, e.g. goal-based scenarios, multi-user construction kits, and computer supported collaborative learning. In order to convey an intended experience, techniques for simplifying the generation of learning systems will accelerate the use of such systems. This is the problem faced by the designers of the on-line museum exhibit current tools do not support the complexity and interplay between levels that allows for deep learning and exploration of content.

Using technology to advance education will require attention to the interplay between local (pertaining to the individual student, subject domain, and classroom) and systemic (pertaining to the broad range of students, all subject domains, and entire schools or school systems) factors or levels in design. In some sense, all learning is local: students construct knowledge in response to the problem at hand, their personal sense-making ability, and the social and technologies supports available. Yet if software design addresses only these local factors, the result is often systemic failure: software that is fragmentary, poorly supported, and easily marginalized. For software to contribute broadly to educational improvement, we must being to address factors which will enable local successes to plug in to larger agendas, scale up to widespread audiences, and evolve to meet new challenges. To address these factors, the research community will have to attend to implementation strategies and, more broadly, software architectures throughout the design process. A software architecture serves as a framework for fitting together components and identifying how different pieces of software (in our case, a learning system) interact with one another. Providing good architectures can make development and implementation easier, including the creation and use of tools for building learning environments.

The need for architecture arises from the inherent complexity of learning processes. Layers of support are needed for:

Broadly speaking, these layers are types of support within learning software. Within these layers, a software architecture for learning systems requires modular objects that provide commonly useful facilities like notebooks, graphs, tables, calculators, mark-up and annotation, and more. Decades of research have taught us much about how an individual layer or object can be designed to enhance learning. Now designs must scale up to suites of layers and objects that offer solutions to learners' overall needs. Three critical challenges to achieving broadly applicable learning software architectures are: Emerging component-based, distributed object architectures offer the potential to overcome these challenges and produce top-quality educational software that can gracefully scale up, spread out, and advance forward. Unlike prior "application island" architecture, components could allow educators to re-use graphs, tables, and other objects rather than re-coding each required capability. Components could support transform educational development into a process of composing, rather than coding. In composing an activity, educators could draw together best-of-class objects and layers, regardless of vendor, and organize their own layout, scripts, and instructional presentation. Distributed objects could realize the promise of the internet and multimedia by making fully interactive, dynamic, collaborative artifacts available to every student regardless of location or platform.

Emerging industry standards of component software might provide a substrate for educational architecture, raising the level of the playing field. However, industry solutions will be aimed first at business needs, and not learning processes. To achieve the potential of educational technology, researchers should consider and utilize the best solutions industry has to offer, and then address industry agendas and techniques to the specific needs of education. The role of the educational technology researcher should not be to simply utilize what's there, but also to direct the process so that the educational agenda is also addressed.

Important research questions for facilitating the growth of educational software architectures include:

3.4 How will the designer evaluate the effectiveness of the design?

3.4 How will the designer evaluate the effectiveness of the design? Assessment and evaluation in the current context of educational technology materials requires the design, development, and implementation of new assessment paradigms. Older frameworks (such as standard software evaluation forms described in OTA's Power on! (US Government, 1988, pages 232-235)) which focus mainly on software attributes, allow for desciptions of software but not of the impact of the software on learners and educators. For design for learners to be successful, assessment must be an integral part of the design and development cycle. Such integration can arise through various mechanisms such as:

New assessment paradigms need to address the impact of software on all stake-holders (from the learner, to the educator, to the larger administration, to the educational process at the highest level) while forming a clear definition on the criteria for success both in the short-term and in the long-term. Such levels of assessment allow evaluation of small changes in the short-term and larger impact effects over time including how applications change the discourse within and among the disciplines. Long term analysis also offers the ability to address the scalability and sustainability of the application. What works well in the small may or may not be viable in terms of cost, production or use in large settings.

The criteria for success must address the effectiveness of use as well as the level of integration ­ software that students hate, software that teachers can't and won't use, and software that teaches things that are not worth the students' time need to be identified by assessment. Definitions of success must allow determinations of motivational impact as well as the quality and integrity of the learning experience. Hard questions need to be asked about the usability and usefulness of the application. Emphasis needs to be placed on deep change as opposed to decorative embellishments.

These new assessment categories demand the establishment of assessment protocols, which combine use of quantitative studies in partnership with ethnographically-based qualitative research. New techniques for capturing and analyzing rich data including information about learner and educators' motivation concerning software use should be encouraged. As we create more materials that encourage and enable learners to generate products, we have to enhance current techniques, or develop new ones, that enable us to judge learner portfolios application,

The education community needs to define metrics and techniques which might be observational, analytic or heuristic for assessing individual learning, group learning, and how this material fits into a social organizational context. Currently, some of the well-defined metrics are for individual performance: time to learn a task, time to complete a project, etc. These metrics are important, but new ones perhaps need to be made and thus we need research in the following areas: