Learning by Design from Theory to Practice


Janet L. Kolodner, David Crismond, Jackie Gray, Jennifer Holbrook, Sadhana Puntambekar

EduTech Institute and College of Computing

Georgia Institute of Technology

Atlanta, GA 30332-0280

{jlk, davidc, grayj, holbrook, sadhana}@cc.gatech.edu



Abstract: In 1996, we reported on a new approach to creating a constructivist learning environment, one that addressed both cognitive and social aspects of learning and that was based on a combination of good theory (taken primarily from case-based reasoning's cognitive model) and a compatible model of practice (problem-based learning) and that incorporated the best of several other approaches (e.g., collaborative learning). Case-based reasoning's cognitive model provided a conceptual framework for envisioning a classroom and curriculum where learning happens as a result of a series of design experiences, problem-based learning told us how to enact exemplary practices in the classroom, and case-based reasoning and its relatives (e.g., analogical reasoning, cognitive flexibility theory) provided specific guidelines for implementing practices. What was needed, we said, was experience in the classroom that we would use to debug the model we had set up. This paper presents Learning by Design (LBD), the approach to practice that we've developed over the past three years based on the guidelines put forth by CBR and PBL. We've learned many different pedagogical tools and practices for accomplishing LBD's goals. Our recent pilots also suggest several key values that must be bought into and practiced by teachers as a necessary condition for successful LBD implementation.


1. Introduction

Too often, science instruction is divorced from students’ engaging interests and the activities of their daily living. Traditional science instruction has tended to exclude students who need to learn from contexts which are real-world, graspable, and self-evidently meaningful. National curriculum reform efforts, including the AAAS’s Benchmarks for Science Literacy and NRC’s National Science Education Standards, are calling for inclusion of technology objectives as a central part of science education and suggest that students should "do science" to gain "lasting knowledge and skills" in design, technology, and the sciences. There are also calls for students to learn complex cognitive, social, and communication skills as part of their middle-school and high-school experiences to help them develop "habits of mind". And there is a need for students to be learning science in ways that allow them to put it into practice in solving problems and making decisions, and not just as inert facts.

With this in mind, Georgia Tech's EduTech Institute decided, in 1994, to use what we knew about cognition to fashion a learning environment appropriate to deeply learning science concepts and skills and their applicability, in parallel with learning cognitive, social, learning, and communication skills. Our intention was to lay the foundation, in middle school, for students to be successful thinkers, learners, and decision-makers throughout their lives, and especially to help them begin to learn the science they need to know to thrive in the modern world (Hmelo et al., 1996).

We took our cues from what we knew about learning and practice, particularly approaches that emphasize learning from problem-solving experience. Case-based reasoning's (CBR) (Kolodner, 1993) cognitive model provided a conceptual framework for envisioning a classroom and curriculum where learning happens as a result of a series of design experiences (Kolodner, 1997); problem-based learning (PBL) (Barrows, 1985) told us how to enact exemplary practices in such a classroom; and case-based reasoning and its relatives (e.g., analogical reasoning (Gick et al., 1984), cognitive flexibility theory (Spiro et al., 1988)) provided specific guidelines for carrying out some of those practices (Kolodner et al., 1996). As well, we brought in findings from research on design cognition, decision making, and collaborative learning. Our intentions are summarized in [Table 1].


• Give students experiences "doing" science, including asking questions, investigating, and applying what they've learned

• Help students learn science concepts deeply by giving them opportunities to discover and confront their conceptions and misconceptions

• Help students learn "key" conceptions and skills that are difficult but essential, e.g., systems thinking and the analysis of invisible and time-delayed causation, models and simulations as ways of finding out

• Help students make connections between their own experiences and science

• Help students make connections between science and the world around them

• Help students learn essential concepts and skills from classroom science experiences

• Motivate students to want to learn and provide them reason to engage

• Help students want to learn and help them understand what it means to learn and takes to learn

• Help students learn life skills as part of their science experiences, especially those that are essential for decision making, justifying, argumentation, and addressing complex problems

Table 1: Learning by Design's Intentions

Our idea was that we would write curricula that would give students the opportunity to encounter design challenges that would serve as compelling contexts for learning science concepts and skills. Such design challenges would provide opportunities for engaging in and learning complex cognitive, social, practical, and communication skills. Construction and trial of real devices would give students the opportunity to experience uses of science and to test their conceptions and discover the bugs and holes in their knowledge. Using the practices of problem-based learning (PBL), the teacher would help students reflect on their experiences in such a way that they would extract and articulate and keep track of both the content and skills they were learning. Using guidelines from case-based reasoning, (i) we would provide libraries of cases to students to use as resources and (ii) teachers would be taught to aim reflective discussions and other activities towards identifying not just what has been learned but also under what conditions it might be applicable and when such conditions were likely to come up in the future. As well, we knew that working on group projects is mired in all sorts of difficulties, so we planned to provide software that would aid record keeping, collaboration, and reflection. [Table 2] summarizes.

• design challenges to promote and focus learning and provide opportunities for application

• problem-based, project-based, hands-on, student-centered

• integrate learning content with learning complex skills

• focus on systems thinking, decision making, application of knowledge and skills

• supply case libraries to provide suggestions and help with focus

• use students' construction failures as opportunities, and iterate on designing, constructing, to test newly-developing conceptions

students discuss a design challenge, design and build, identify what they need to learn, carry out investigations, design and build again, and so on

• interleave doing and reflection

intersperse doing with well-facilitated and orchestrated reflection in an effort to help students turn their experiences into accessible, reusable cases

• software in support of collaboration, record keeping, reflection

Table 2: Learning by Design: Early Conceptions

We reported, in 1996, that several iterations would be needed before we would be able to turn our ideas into well-articulated curricula and principles of practice. Two years later, we are far along that path. Learning by Design is more sophisticated now than it was in 1996. It addresses student and teacher needs better. It is also far more structured and focused than we could have imagined. On the other hand, it is far more learner-centered than it was, focusing more on student and teacher development and on the social structure and culture of the classroom than it did back in 1996.

2. Learning by Design: Essential Components

The typical sequence of activities in a Learning-by-Design unit has students encountering a design challenge and attempting a solution using only prior knowledge -- individually and/or in small groups. In whole-class discussions, the teacher helps students compare and contrast their ideas, identify what they need to learn to move forward in addressing the design challenge, choose a learning issue to focus on, and design and/or run a laboratory activity to examine that issue. This discussion provides an opportunity for the teacher to identify student misunderstandings and misconceptions and begin the process of supporting those. The teacher might also present demonstrations, assign readings, and/or present short lessons relevant to discovered knowledge gaps. Following this are cycles of exploratory and experimental work, followed by reflection on what has been learned, application of what was learned to achieving the design challenge, evaluation of that application, and generation of additional learning issues. Potential solutions to the design challenge are attempted in each cycle and evaluated by building and testing a model or actual device; comparing different design alternatives based on qualitative and/or quantitative understandings; or analyzing using established design guidelines or the ratings of experts. Within this cycle are several opportunities for students to share their work with others and hear their feedback and ideas. Important during these "gallery walks" and "pin-up sessions" is that students justify their design decisions and explain how their designs work (or would work) using science and engineering vocabulary.

We've written one long (8-week) content unit and tried out many shorter (2- to 6-week) LBD-inspired problems, including Vehicles in Motion, an 8-week unit in which students learn about forces and motion by redesigning vehicles and their propulsion systems; Jekyll Island, a 4-week problem, where students learn about erosion and water by coming up with and modeling means of combating the beach erosion that is threatening the island's existence; and Rome/Savannah Subway, a 6-week unit that helps students learn about different kinds of rocks and rock formations, and their implications in the context of designing an imaginary subway system.

2.1 A Sample Module: Vehicles in Motion

Vehicles in Motion (ViM) has been through four iterations, and field testing begins in August, 1998, in preparation for dissemination. Students are challenged to design mechanically-powered toy vehicles that can carry a load over a hilly terrain. Students we've worked with (most recently, 250 spread among 4 teachers in 4 local middle schools) have engaged with the challenge, enjoyed it, and learned (internal documents), even in classes where the teachers were unsure about making LBD work.

Students get started experimenting with toy cars and other toy vehicles that achieve or fall short of achieving the challenge. They begin to notice the forces needed to get vehicles moving and some of the differences between toys that move easily and those that don't. They begin to ask questions: How can I get a car started? Why does this one start more easily than that one? How can I give this car more power to go over the bumps? How can I keep it going after it goes over a bump? Is there a way I can control its speed? And so on.

Individual and small-group exploration is followed by whole-class discussion where students share their discoveries and questions with each other and begin to consider what it would take to achieve the challenge. Discoveries and questions are captured on communal whiteboards and in students' design diaries. Students have different ideas about the answers to the questions and different ideas about how to proceed, and those differences become a driving force in moving the learning forward. The teacher helps students recognize that deciding between their answers requires some knowledge they don't yet know. This promotes a need to investigate, experiment, read, and explore. The teacher help students turn their original design and construction questions into questions about friction, inertia, mass, gravity, speed, and other concepts associated with motion and forces.

The unit includes investigative modules for each fundamental concept. The friction module has the students begin their construction activities. They build a vehicle (using our instructions) whose bearings are not optimal, and they experience the effects of friction on the workings of their vehicles. They then experiment with different kinds of bearings and redesign their bearing systems so that their vehicle can coast well. They also work on measuring, graphing, and "fair tests" in this module. There are also several activities independent of the vehicle challenge that allow them to experience and measure effects of friction and ways of alleviating it.

A second module takes students through investigations with a balloon-powered propulsion system. They learn how to measure speed and what acceleration is, and they get experience designing and running simple investigative experiments aimed at optimizing the balloon-powered system for carrying the greatest load the greatest distance.

We then have them try their optimized balloon-powered vehicle on the challenge track, where they discover a need for other kinds of propulsion systems -- ones that can exert greater force at the start (to overcome inertia) than the balloon-powered system. We give them instructions for constructing two other propulsion systems (rubber-band and falling-mass powered), each with different behavior, and they repeat their investigations with these, ultimately designing and building a hybrid propulsion system that can achieve the challenge.

Each module explores a set of issues that have already been identified by the children as important to achieving the design challenge. Each includes investigation with the vehicles they are constructing, other activities that allow them to experience the concepts being explored, and opportunities to redesign or optimize their vehicles or propulsion systems. Several times during each module, students share their design ideas and what they've discovered in investigations with their peers, always using the data they are collecting as evidence to justify design decisions they are making. As well, students keep records of their ideas, the experiments they are doing and the data they are collecting, and the science they are learning in their design diaries (Puntambekar & Kolodner, 1998). They jot down notes during class as they are working and as whole-class discussions are going on, and for homework, they write up in better form the things they are learning about science concepts and science and design skills. Classroom discussions focus on the design challenge, the science that is being learned, and the skills students are engaged in (e.g., experiment design, decision making, teamwork). In the last module, students work towards integrating what they've learned in the context of refining and merging ideas from the three propulsion systems to design one that can achieve the design challenge. They write it up and present it.


2.2 Helping students gain the skills for LBD

Each of our modules and units involves experimentation, decision making, collaboration, good use of evidence, and other complex skills. We've learned that students and teachers need experience with the essentials of doing science and design before they are comfortable learning science from design activities. This has led us to write a 3-week-long "launcher" unit, called Apollo 13, that introduces students to making decisions, understanding and discussing devices, constructing, collaborating, reflecting, keeping records, and designing and running experiments. That unit uses the episode of launching and rescuing the Apollo 13 mission as an example and motivator of the relevance of science knowledge and skills and the need to learn such skills as decision making, collaboration, argumentation, use of evidence, and so on that are essential within and outside of science and critical to learning science through design activities.


2.3 Critical components of LBD

Experimenting with LBD-inspired problems in real classrooms with real teachers has led us to a deeper understanding of its critical components. For example, before we knew which parts of LBD were essential to success, we tried out several problems where it was difficult for students to build working models of their designed devices and where teachers implemented the design component as an end-of-the unit activity rather than iterating on design (Hmelo et al., 1997, Gertzman & Kolodner, 1996). In a later trial, where we integrated designing with investigation and where we made sure that students were constructing and testing the devices they were designing, we found that construction took too much time, that it required "authentic" materials, and that teachers needed to understand the underlying science well to be successful facilitators. From these, we collected evidence that several principles of LBD are essential, e.g., constructing and testing designed devices, multiple design iterations, design interleaved with investigation, whole-class sharing and critiquing of ideas, and some means of orchestrating reflection. We learned as well about help that students need to be successful designers (Puntambekar & Kolodner, 1998). The more mature conception of Learning by Design that came from those experiences has many key components:

a. Authentic, engaging design-and-build activities that enliven students’ interest in science.

b. Case-based reasoning [Kolodner, 1993] and use of analogical reasoning with concrete cases to inform design decisions (e.g., experience with toy cars, paper-based cases, on-line case libraries).

c. Multiple contexts for design activities, including designing devices, fair-test experiments, instruments for measurement, and information searches;

d. A balance of constrained, scaffolded challenges with more open-ended design tasks. Naive and novice designers benefit from tasks like product redesign (Crismond, 1997).

e. Rich varied feedback for designers, through real-world testing, peer and expert analysis of conceptual designs, comparisons to expert model-case solutions and comparisons between peer designs, and extant expert design guidelines.

f. Well-orchestrated approaches to generating classroom discussions and collaborative work, including gallery walks and pin-up sessions for studio-like design discussions (Schön, 1991) and work with white boards to record and support student-generated inquiry (as in "problem-based learning", (Barrows, 1985)).

g. Experimental and exploratory laboratory work that supports the design challenge and engages students in investigation. Many times students design and create ways of exploring key science concepts.

h. Support for process thinking and self-monitoring and a strong reflective component, through the use of design diaries, "design dials", reflective discussions, publication of lessons learned, and optional sophisticated, but easy-to-use, software in support of the wide variety of learning, problem-solving, discourse, and understanding activities that students engage in.

i. "Launcher" activities to introduce students to the science skills and complex cognitive and social skills critical to doing, learning, and applying science.


3. Making it Work: Scaffolding Student Cognitive Development

But simply including these kinds of activities in a science unit doesn't guarantee learning. Important as well is that they be well-integrated and that students receive the help they need to be successful in all of what they are doing. Students need support of several kinds to engage successfully in these endeavors (Puntambekar & Kolodner, 1998): help with managing investigation, experimentation, design, and their many parts; coming up with ideas, evaluating, deciding what to try or do next, accessing resource materials, and so on. They also need help with reflecting on their experiences so as to recognize and be able to articulate the content and skills they are learning.

Our approach integrates activities into several cycles and makes several suggestions about scaffolding. We aim for small groups to be successful at achieving viable solutions to design challenges and for individual students to learn content and skills.

• A variety of types of activities cement learning and provide focus on science. (i) Initial construction and trial are aimed toward promoting engagement, use of prior knowledge, and inquiry. (ii) Exploratory and experimental laboratory activities are aimed toward discovery and understanding of science concepts and skills. (iii) Application activities are aimed toward broadening of science concepts over a range of situations. (iv) Reflection is aimed at promoting articulation of concepts and skills being learned and their integration with concepts and skills learned earlier.

• Individual activities are connected to each other in cycles that promote drawing connections between design challenges and science. (i) The research cycle connects a design challenge to what needs to be learned to solve it, learning of those things during library research and laboratory activities, articulation of what is learned, application, and further questioning. (ii) Pin-up sessions provide opportunities for small groups to share their ideas with the larger class and to hear comments and suggestions. Students are asked to justify their decisions and ideas in terms of science concepts and experimental results. (iii) A reflection cycle asks students, as individuals, to reflect on and articulate the content or skills they are learning; asks students to share their reflections with the class in an orchestrated discussion; optionally includes further small-group discussion; and then asks for written individual articulation of what has been learned. (iv) To help students learn the full complexity of science concepts and their inter-relationships with each other, students (a) experience the complexity first and based on that, ask questions that get at its component parts, (b) experience and experiment with each of those component concepts separately in a unit's modules, and (c) then put it all back together in the design challenge, all the while keeping the big picture in mind as they are working on the pieces.

• A variety of paper-and-pencil and blackboard-based methodologies help students keep records, prompt at appropriate times, and promote reflection and articulation. (i) PBL-inspired whiteboards (Barrows, 1986) provide means of recording the high points of whole-class and small-group discussions, especially focusing on what's known about a design challenge and its environment, ideas about how to address the challenge, and issues that are important to learn more about. (ii) A "design dial" helps student groups keep track of the phase of designing they are engaged in and its key components. (iii) Design diaries prompt individual students to record their ideas, to identify the science they are learning and need to learn more about, to note ways of testing their designs, and to record and justify conclusions and prompt small groups to keep track of hypotheses, to record the results of trials, and to record the ideas of the group.

• Teacher materials specify learning objectives and goals of each activity; help teachers anticipate difficulties in understanding that will arise and provide advice about dealing with each; point out vocabulary that can be introduced and the opportunities that might be found for introducing it; inform on how to integrate design diaries and other scaffolding into the proceedings; provide activity sheets for students; and suggest rubrics for assessing student written and oral work so as to be able to give students feedback, to assess their progress, and to give them grades.

• Optional software tools (Guzdial et al., 1996) enhance all of this, complementing the teacher's efforts and the paper-and-pencil tools. We've implemented sharable whiteboards, on-line collaborative forums for sharing and critiquing design ideas, on-line case libraries as resources (Narayanan et al., 1996), and case-authoring tools (Shabo et al., 1997) that prompt students to publish what they are learning for other students to benefit from.


4. Unanticipated Logistical and Cultural Challenges

We didn't know, back in 1995 and 1996, how long the distance would be between having a "good" curriculum idea and putting it into practice successfully. Our LBD units and approach seem to address cognitive issues relatively successfully. But we've discovered many challenges -- developmental, social, logistical, and cultural -- that make must be taken into account for success.

• Teachers are neither expert designers nor expert facilitators; there is a need to build much scaffolding into the curriculum and materials to complement teacher skills. Our toolbox of classroom activities and sequences of activities that promote question asking, summarization, reflection, and so on, are aimed toward that. But they aren't enough.

• Students are not adept or experienced at group work, decision making, experimentation, or use of evidence, and it takes time for them to develop those skills. The 3-week launcher unit gets them started. But they also need guidance as they are developing those skills over time.

• Kids (and some teachers) don't know how to construct. In fact, sometimes they don't even know how to examine a device and comment on what they see.

• Teachers are not familiar enough with science content.

• Neither kids nor teachers are used to active learning or to real challenges.

• "Chaos" in the classroom makes it hard on the teacher (and on others down the hall). Kids aren't used to the freedom of the LBD classroom, and teachers are dismayed by the "chaos."

We are addressing these challenges through a developmental approach for teachers and students, (i) making explicit for both what might be difficult, (ii) laying out developmental progressions in skill acquisition, (iii) identifying what's essential for getting started, (iv) suggesting activity structures for the classroom that scaffold both teachers and students, (v) providing scaffolding for students that also scaffolds teachers, and (vi) providing forums for teachers and students to discuss what's happening in class with each other.

We are implementing this approach through several mechanisms: (i) We think that if students have ownership (in a student book) of most of the same materials teachers have, then the dynamic in the classroom can be made such that students can contribute to classroom conversations in ways that scaffold teacher skills and teacher knowledge. (ii) We are writing guidelines and rubrics for students to use to help them understand the ins and outs of skills they are using and to enable them to recognize what their capabilities are and where their next developmental steps might be directed. (iii) We've written materials to help with construction (e.g, fact sheets on concepts like "clearance," "attaching things," and "recognizing obstructions"), and we've redesigned design activities so that students are, in general, redesigning. They build first based on construction guidelines and then focus their design decision-making in critical areas. This way less time is "lost" on unrelated construction and trials. (iv) Ongoing teacher professional development activities allow teachers to learn through an LBD approach and continue to help each other -- summer workshops form a beginning; year-round on-line discussion forums allow teachers to share frustrations and ideas and gain growing confidence; periodic meetings during the year allow problems to be solved together.

Most important, though, we've found, is the classroom climate. In classes where our teachers have helped students learn, from the beginning, that they are all responsible for each others' learning, where the teacher makes clear that he/she respects the students as learning partners, where the teacher engages with the students in addressing the design challenge, and where the teacher insists on respect towards everyone, the students respond quickly, engage enthusiastically, and learn much, even when teacher skills and knowledge are deficient. When the teacher goes through the motions on all of the activities but makes no changes in the classroom climate, the students don't engage well either.

With all of this in place, we are hoping that teachers who want to make LBD work will be able to do it, despite lack of knowledge and/or skills. Our experiences with teacher development this summer suggest that motivated teachers will be able to be successful, but of course, this remains to be seen.

5. References

[Barrows, 1985] Barrows, H. S. (1985). How to design a problem-based curriculum for the preclinical years. NY: Springer.

[Crismond, 1997] Crismond, D. (1997). Investigate-and-Redesign Tasks as a Context for Learning and Doing Science and Technology: A study of naive, novice and expert high school and adult designers doing product comparisons and redesign tasks. Unpublished doctoral thesis. Cambridge, MA: Harvard Graduate School of Education.

[Gertzman & Kolodner, 1996] Gertzman, A. & Kolodner, J. L. (1996). A Case Study of Problem-Based Learning in a Middle-School Science Class: Lessons Learned. Proceedings of ICLS '96 , Charlottesville, VA: AACE.

[Gick & Holyoak, 1980] Gick, M. & Holyoak, K. J. (1980). Analogical Problem Solving. Cognitive Psychology 12: 306-355.

[Guzdial et al., 1996] Guzdial, M., Kolodner, J.L., Hmelo, C., Narayanan, H., Carlson, D, Rapping, N., Hubscher, R., Turns, J., & Newstetter, W. (1996). Computer Support for Learning Through Complex Problem-Solving. CACM, Vol. 40, pp. 39-42.

[Guzdial et al., 1997] Guzdial, M., Hmelo, C., Hubscher, R., Nagel, K., Newstetter, W., Puntambekar, S., Shabo, A., Turns, J. & Kolodner, J.L. (1997). Integrating and Guiding Collaboration: Lessons Learned in Computer-Supported Collaborative Learning Research at Georgia Tech. Proceedings Computer Support for Collaborative Learning '97.

[Hmelo et al., 1996] Hmelo, C.E., Holton, D.L., Allen, J.K., Kolodner, J.L. (1996). Designing for understanding: Children's lung models. Proceedings of the Eighteenth Annual Conference of Cognitive Science Society, LEA, pp. 298-303.

[Holyoak & Thagard, 1997] Holyoak, K. J. & Thagard, P. (1997). The Analogical Mind. American Psychologist, Vol. 52, No. 1.

[Kolodner, 1993] Kolodner, J.L. (1993) Case-Based Reasoning. Morgan Kaufman Publishers, Inc., San Mateo, CA., 1993.

[Kolodner, 1997] Kolodner, J.L. (1997). Educational Implications of Analogy: A View from Case-Based Reasoning. American Psychologist, Vol. 52, No. 1, pp. 57-66.

[Kolodner et al., 1996] Kolodner, J.L., Hmelo, C.E., & Narayanan, N.H. (1996). Problem-based Learning Meets Case-based Reasoning. In D.C. Edelson & E.A. Domeshek (Eds.), Proceedings of ICLS '96 , Charlottesville, VA: AACE, July, 1996.

[Narayanan et al., 1996] Narayanan, N. H., Hmelo, C.E., Holton, D.L. & Kolodner, J.L. (1996). Case Libraries for Middle School Science Instruction. Proceedings of ICLS '96 , Charlottesville, VA: AACE, July, 1996, p. 566.

[Puntambekar & Kolodner, 1988] Puntambekar , S. & Kolodner, J. L., 1998. �The Design Diary: A Tool to Support Students in Learning Science by Design. this volume.

[Shabo et al., 1007] Shabo, A., Nagel, K., Guzdial, M. & Kolodner, J.L. (1997). JavaCAP: A Collaborative Case Authoring Program on the WWW. Proceedings Computer Support for Collaborative Learning '97, pp. 241-249.

[Spiro et al., 1988] Spiro, R. J., Coulsen, R. L., Feltovich, P. J., & Anderson, D. K. (1988). Cognitive flexibility theory: Advanced knowledge acquisition in ill-structured domains. In Proceedings of the Tenth Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.


This research has been supported in part by the National Science Foundation (ESI-9553583), the McDonnell Foundation, the BellSouth Foundation, and the EduTech Institute (with funding from the Woodruff Foundation). The views expressed are those of the authors.