The Design Diary: A Tool to Support Students in Learning Science by Design

Sadhana Puntambekar

Information Media and Technology

School of Education, U-64

249 Glenbrook Road

University of Connecticut

Storrs CT 06269-2004

Janet L. Kolodner

EduTech Institute

Georgia Institute of Technology

250 14th Street, NW, SUITE 143

Atlanta, GA 30318

Abstract: We describe the design and development of design diaries, a tool for scaffolding learning by design. The diaries have prompts in them to help students carry out their design related activities. We used the diaries to support life, earth and physical science problems. We will discuss how we made the prompts in the dairies progressively more specific, based on our classroom experiences, and a more detailed model of the design process.



1. Introduction

Many in science education are focusing on introducing more hands-on science activities into classrooms (Hameyer, Akker, Anderson and Ekholm, 1995); students learn science by "doing" science and "using" science to solve problems or explain phenomena. But effective implementation of such activities is not easy. The challenge of increasing students’ understanding of science cannot be met by merely throwing more science facts and principles at students, increasing laboratory activities, or simply emphasizing "hands-on" activities. Such activities effectively teach science only when embedded in a "minds-on" (Glynn, Yeany and Britton, 1991) learning environment in which students are encouraged and helped to develop both deep understanding of science concepts and skills of problem solving and reflective thinking Rakaw (1985).

Integrating minds-on design activities into the curriculum provides a powerful way of creating such a learning environment (e.g. Harel 1991; Kafai, 1994; Roth, 1996; Lehrer & Romberg, 1996, Kolodner, 1997). Design provides a natural environment for discovering reasons why science concepts need to be learned, for seeing the contexts in which those concepts are put to use, for using those concepts to solve meaningful problems, and for engaging in meaningful reflection and thinking. Design integrates many mental activities that are often fragmented across the curriculum (Lehrer & Romberg, 1996). Design activities that require construction of a working artifact provide ongoing feedback for students as they confront their understanding of concepts by trying to put them into practice. The process of designing affords opportunities for students to incrementally construct, evaluate, discuss and revise both the models they are designing and their conceptions (Penner, Lehrer, & Shauble, 1996).

However, implementing design activities in the classroom is not easy. For effective learning to happen, it is important that design be used as a vehicle for learning and that discussions concentrate on the process not just the end product of the design activity (Roth, 1996). Design activities need to include multiple iterations of constructing, evaluating, and revising artifacts or models, along with discussion and articulation of issues and principles that arise in solving the design problem. Used this way, design activities become an excellent vehicle both for engaging students in wondering about and exploring relevant science, math, and technology concepts and for learning mental skills that are critical for real-world problem solving.

The goal of Georgia Tech's Learning by Design project is to help middle school children learn science content more deeply and at the same time develop the skills and understanding needed to undertake the solution of complex, ill-structured problems. Our hypothesis is that this can be accomplished by situating science learning in design problem solving. In our first implementations, we trained teachers at being facilitators and helped them learn about the particular design problems their students would work on. Based on our analyses of these experiences, we discovered some of the specific difficulties students had and developed paper-and-pencil scaffolding tool, the design diaries, to address some of these difficulties. Experience with the design diaries showed us just how much more fine-grained and problem-specific the scaffolding needs to be.


2. First Implementations

Although design is an effective medium to help students learn science, learning from design activities is complex for middle schoolers. Our pilot studies helped us understand that students need support in carrying out each of the stages of designing. We conducted two pilot studies. In the first study, students participated in a two week unit on arthropods -- they learned about arthropods by designing and building a model a robot with features of arthropods. In the second study, students were asked to design a lung.

Much of the scaffolding provided during these activities was based on the Problem-based Learning (Barrows, 1985) approach. Teachers took the roles of facilitators, both drawing students out and summarizing important aspects of the discussions. Whole-class discussions focused on identifying what students already knew, issues they needed to learn more about, ideas for solving the problem, and plans for moving forward. Notes from these discussions were recorded on PBL-inspired white-boards that were posted for all students to see. Design activities themselves happened in teams of four students. Teams were given information about the design process emphasizing the design questions that they should consider as they went about designing. For example, those designing arthropod-inspired robots were asked to think about the features of arthropods, what function their model would serve, what their model would look like, and who would use it. They were also given handouts containing information on making moving limbs, joints, etc.

We found that students teams needed explicit guidance to use the information on the white boards, and they did not use that information unless they were explicitly asked to do so. They needed support to understand what features to include in their models, to research the issues they had identified, and to relate them to their models (Gray, Young & Newstetter, 1997). Students needed explicit guidance in understanding the problem, carrying out the research, building and evaluating their prototypes, and in integrating the science with their design activities. We also found that students needed to go through cycles of modeling and design to develop a deeper understanding of the science concepts. But just going through such iteration by itself is unlikely to be optimal unless scaffolding is provided to facilitate students’ learning (Hmelo, Holton, & Gertzman, 1997).


3. First Iteration of Systematic Scaffolding: The Design Diaries

3. 1 The Scaffolding

Based on these findings, we developed and implemented a first approach to scaffolding for students. Scaffolding was in the form of design diaries (Puntambekar, 1997), a paper and pencil scaffolding tool. The diaries served many functions. Based on the notion that "making covert, abstract processes visible, public and manipulable, serves as a necessary catalyst for reflective metacognitive activity" (Derry, Tookey & Chiffy, 1994), the diaries served as a vehicle for providing hints. They helped students recognize what phase of designing they were in and record ideas and knowledge relevant to that phase of designing. They also provided prompts to help students decide how to move forward. They thus made thinking visible and recorded students’ journey through the design process. They contained prompts associated with each of the major stages of designing (problem understanding, information gathering, choosing a solution and evaluation), which students used as they were engaging in design activities in the classroom, and as they reflected on those activities at home in the evening. The diaries were used for an implementation of the lung problem in the same classrooms that had already done the arthropod problem. Students were required to build models to show the working of the breathing system. Models needed to show the lungs and one other part of the system.

Design diaries provided guidance for students both in carrying out design activities and reflecting on them in order to learn from them. They were used as a vehicle for providing hints, usually in the form of leading questions, as students were engaging in design activities and as they reflected on those activities at home in the evening. Diaries prompted students to make to make critical decisions, such as what function would they model, what materials they would use and why, what they needed to learn to model the function they chose, how they would evaluate potential solutions, whether they needed to revise their models, and if so, why. The main phases of the design process -- problem understanding, investigation, generating alternatives, and choosing a solution -- were supported.


3.2 What We Learned

Students’ responses in the design diaries showed that they required support at a more fine-grained level than we provided. While we provided support for the critical phases of the process, we provided little support for the individual activities within each phase. For example, although the diaries provided prompts to help students understand the problem better, they did not provide explicit support to help them specify the purpose of their models and come up with appropriate learning issues based on that. Further, we did not support the many activities in the solution generation phase, such as generating criteria, evaluating solutions with respect to these criteria, etc. As such students still had many difficulties with the activities within each of the design phases. We found that students on the whole were unable to specify the purpose of their designs. They concentrated on gathering interesting facts and did not focus their research on the issues that would help them design. They were not very good at generating alternative solutions. Even when they did generate solutions, because the diaries did not support generation of criteria and evaluation of their solutions, they used only pragmatic constraints (such as time, availability of materials) to judge the appropriateness of their solutions.


4. Second Iteration of Design Diaries

4. 1 The Scaffolding

Based on the above findings, we designed a new set of diaries that supported not only the main phases of design, but also the activities in each phase. We also provided students with prompts to help them monitor their learning, for example, to look back at the learning issues while they did the research.

We supported four main stages of the process -- analysis or problem understanding, exploration, solution generation, and evaluation. Each stage consists of numerous activities. For example, the problem understanding phase consists of activities such as identifying the objectives, breaking the problem into subproblems, writing an initial specification and coming up with initial ideas. The exploration stage involves further understanding of the problem, narrowing down a set of issues to learn more about, literature search, refining the problem specification, establishing criteria for evaluation and other activities that help in clarifying the situation. In the third stage, that of solution generation, the knowledge for the earlier two stages is combined. It is the stage in which criteria are set, alternatives are put forth, and a solution is chosen. In the final stage, evaluation, a prototype is built and evaluated with respect to the criteria that have been generated. This is followed by identifying aspects for improvement and redesign. Table 1 shows some of the example prompts from the diaries.

These diaries were used in two situations: To support two earth science problems and two life sciences problems. The results that we are discussing in this paper are from one of the earth science problems that we supported. This problem required students to design ways to stop the erosion on Jekyll Island, one of the Barrier Islands off the coast of Georgia.


4.2 What We Learned

Student responses in the diaries were analyzed for the depth of their science understanding as revealed in the various design activities that they carried out. One of the most important findings was that it was not easy for students to relate their design activities to the science that they were learning. Very often their responses were too general and did not reveal an understanding of the science concepts involved. For example, in the Jekyll island problem, very few students explicitly elaborated the problem statement to indicate that the wave currents are a cause of erosion and they needed to do something about the currents. Most students only showed a shallow understanding of the problem, such as "we need to do something about people losing their properties on Jekyll". Thus they were unable to pin down the purpose that their designs had to fulfill. Many of them were more concerned about effects their designs would have on the environment than the problem of controlling erosion. Not surprisingly, their learning issues were shallow too, and they did not get to the deep science issues about long shore currents, tides, and erosion.

Design phase

Activity supported

Example prompts

problem understanding

Generating question

to understand problem

This is what I understand of the problem (Please restate the problem in your own words)

What questions do I need to ask in order to understand the problem better?

Generating solution

Evaluating alternative


What are the problem requirements?

What are the criteria?

Which criteria do each of your solutions meet?

Which do they not meet?

What are the positive features of each of the solutions?

What are the limitations of each of the solutions?

Generating solution

Coming up with criteria

What are the criteria against which you will evaluate possible solutions? Why do you think these criteria are important?


Table 1: Example prompts from second version of diaries

Most often, the questions that they raised dealt with issues such as "How do we build a sea wall" or "How can we stop erosion". Some of the students raised questions that made them look into the causes of erosion -- in other words, the how questions. As many of the students did not relate the problem on Jekyll to the science they were learning, their initial solution ideas were rather vague, sometimes even outrageous (e.g., put houses on stilts). However, as they progressed through the problem, some of these same students did improve their understanding of the issues concerned, beginning to mention concepts such as blocking or slowing down the current.

As their elaboration of the problem did not reflect an understanding of the science, the criteria that they used to evaluate their designs were also too broad. Very often, these related to designing and building rather than the science. For example, students came up with criteria such as ‘cost’, ‘will it last’, and ‘what will be its effect on environment’. Very few students listed criteria such ‘will it slow down the current’.

Analysis of the second version of the diaries helped us understand that we needed to provide prompts that will ‘elicit’ students’ understanding of science issues. Unless such support was provided, students did not really solve the problem with the deliberate intention of understanding the science. They simply built their models, tested them and made the statement that ‘it worked’. They are unable to justify any of their decisions in terms of the important science issues. We found that not only do students need more specific prompts during each of the design activities, the prompts need to be such that they will encourage students to explore the science.


5. Third Iteration of Design Diaries

5. 1 The Scaffolding

Based on these findings, the third version of design diaries adds more support for linking design to science. Table 2 compares prompts from the second and third versions. Note that the prompts are far more specific in version three. They encourage students to reason about the purpose of their designs right from the start, thus having them think about the science issues that they should be addressing. They need to justify why the criteria that they generated are important to the solutions. Prompts are also provided to help students go back to what they have already written in the diaries, thus helping them monitor their learning. For example, they are asked to read their problem specification again before they generate criteria. In addition, we have also added examples for each of the activities in the design process.

Design phase

Activity supported

Example prompts from version 2

Example prompts from

version 3

problem understanding

Generating question to

understand problem

This is what I understand of the problem (Please restate the problem in your own words)

What questions do I need to ask in order to understand the problem better?

Write a brief statement of the problem emphasizing the purpose of the design.

Questions you might answer

in your statement.

What is the purpose / function of the design?

Where will it be used?

Who will use it?

What exactly are you

supposed to design?

A statement of the problem should not be too vague or

too detailed.


Generating solution

Evaluating alternative


What are the problem requirements?

What are the criteria?

Which criteria do each of your solutions meet?

Which do they not meet?

What are the positive features of each of the solutions?

What are the limitations of each of the solutions?

For each solution write

What is good about it

you should include the criteria that the solution satisfies (from

the criteria page)

you may want to list other criteria as well, and may also want to update the criteria page

What is not so good about it

you might use the criteria that the solution does not satisfy (from the criteria page)

you may want to list other criteria as well, and may also want to update the criteria page

What is interesting about it

you might include what is unique or interesting or

special about this solution

Generating solution

Coming up with


What are the criteria against which you will evaluate possible solutions? Why do you think these criteria are important?

Discuss possible problems and advantages such as

can it achieve its purpose?

what characteristics does it need to have to achieve its purpose?

how long it takes to make

how durable it is

How does it work in the environment - is it safe, is it easy to use

Use this list to set up evaluation criteria.

Use the evaluation criteria to judge your

design on the next page

Read your specification again

to help generate the criteria

Table 2 - prompts from version 2 and version 3 of design diaries

5.2 What We are Learning?

We analyzed student responses for the depth of science understanding. We found that in contrast to the earlier diaries (second version), students’ responses in the new version showed a greater understanding of the science issues in the Jekyll problem. Many of the students explicitly mentioned the causes of erosion right from the start of the problem. Consequently their solutions were more focused towards finding ways to slow down the long shore current and some of the other factors that are unique to the Jekyll island. Many of the students also progressed from the ‘ridiculous’ to the ‘realistic’ in their solution ideas. For example, one group of students very strongly believed that a ‘biodome’ is the only answer to the Jekyll problem. However, as they progressed through the early stages of design, the solution they came up with was to build ‘jetties’. In addition, some of the students got a more realistic appreciation of the problem in that they mentioned that they could not stop the erosion completely, but could only slow it down. Thus the prompts in the third version of the diaries were more successful in getting students to think about the science issues.


6. Are We There Yet?

We are closer, but our observations have shown us that there are several things that we need teacher support for. Our analyses have helped us to make the scaffolding more specific, so that the design activities lead to an understanding of the science. First, we need more whole class discussions with the teacher as the facilitator, to help students tackle science issues. Second, the diaries need to be used as part of a system of scaffolding, so that some of the support is provided by expert facilitation by teachers.

7. References

Barrows, H. S. (1985). How to design a problem based curriculum for the preclinical years. Springer-Verlag: NY.

Derry, S., Tookey, K., & Chiffy, A. (1994). A microanalysis of pair problem solving with and without a computer tool. Paper presented at the Annual Meeting of the American Educational Research association, New Orleans, LA.

Glynn, S. M, Yeany, R. H., Britton, B. K. (1991). A constructive view of learning science. In S. M. Glynn, R. H. Yeany, & B. K. Britton (Eds.), The Psychology of learning Science. Erlbaum: Hillsdale NJ.

Gray, J., Young, J. & Newstetter, W. (1997). Learning science by designing Robots: Knowledge acquisition about arthropods and collaborative skills development by middle school students. Presented at AERA 1997.

Hameyer, U., Akker, J., Anderson, R. D. & Ekholm, M. (1995). Portraits of productive schools: An international study of institutionalising Activity-based practices in Elementary Science. SUNY Press: Albany.

Harel, I. (1991). Children designers. Ablex: New York.

Hmelo, C., Allen, J. & Holton, D. & Kolodner, J. L. (1997). Designing for understanding: Children’s lung models. In proceedings of the Annual meeting of the Cognitive Science Society, pp. 298-303.

Kafai, Y. B. (1994). Minds in play: Computer game design as a context for children's learning. Erlbaum: Hillsdale NJ.

Lehrer, R. & Romberg, T. (1996), Exploring children's data modeling. Cognition and Instruction, 14(1), pp. 69 - 108.

Penner, D. E. Lehrer, R. & Shauble, L. (1996). From physical models to biomechanics: A design based modeling approach.

Puntambekar, S. (1997). Supporting the design process by using design diaries in the ‘Learning by Design’ environment. Paper Presented at the AERA annual meeting, March 24-28, Chicago.

Roth, W. -M, (1996). Art and artifact of children's designing: a situated cognition perspective. In The Journal of the Learning Sciences, 5, (2), pp. 129-166.


This research has been supported in part by the National Science Foundation (ESI-9553583), the McDonnell Foundation, the BellSouth Foundation, and the EduTech Institute (with funding from the Woodruff Foundation). The views expressed are those of the authors.