I. Characteristics of the functional programming paradigm
As we've mentioned early on, we'll be working in the functional
programming paradigm for several weeks. We talked a little bit
about the constraints this paradigm places on you, the functional
programmer, but now's a good time to talk about them in a bit
more detail.
In general, the functional programming paradigm requires that the
programmer avoid the use of what computer folks call "side effects".
A side effect is anything that a function does that persists after
the function has stopped executing. So strict adherence to this
basic premise of functional programming means that we can use
functions that return values to other functions so long as they don't
leave other stuff lying around as a side effect. The primary impact
on us is we can't create functions that store values in additional
memory locations called "variables", nor can we use pre-existing
functions that store values in variables. Storing values in variables
is called "assignment", so we can't do assignment yet. In fact, we
can't use variables at all...there's no need to if we can't store
stuff in them.
And because there are no variables and no assignment, we can't use
traditional methods of getting repetitive operation out of our
functions...repetitive operation is called "iteration", and standard
forms of iteration use what are commonly called "loops". Since you
have previous programming experience, you probably know what a loop
is, and you may already miss it. All we have for the time being is
recursion, however, since we haven't introduced all the extra
baggage necessary for making loops happen. So get comfortable
with recursion.
In the domain of the Scheme programming language, what all this
means is that you can't use Scheme's predefined functions that cause
side effects. These functions include "let" and "set!" Stay away
from those for now; we'll let you use them later. Note that "define"
also has a side effect; it assigns a function body to a function
name. We can't get around that one...we don't have another way to
define functions. But it's also possible to use define to create
variables, and we don't want you to use define in that way now. Why
are we putting these constraints on you? We want to keep the
language you need to know as small and simple as we possibly can in
the early going so that we can concentrate on other stuff; we'll let
you make things more complicated as you gain more experience.
II. The substitution model of evaluation
Let's go back and revisit the factorial function again:
(define (factorial n)
(if (= n 0)
1
(* n (factorial (- n 1))) ))
To explain how the factorial function computed a result Tuesday, we
quickly sketched out something like this (but maybe not exactly like this)
on the whiteboard:
(factorial 3)
(* 3 (factorial 2))
(* 3 (* 2 (factorial 1)))
(* 3 (* 2 (* 1 (factorial 0))))
(* 3 (* 2 (* 1 1)))
(* 3 (* 2 1))
(* 3 2)
6
This is a shorthand way of saying the following:
The original call of (factorial 3) can be replaced (or substituted
for) by the body of the code in the function definition. Thus,
(factorial 3)
can be replaced by
(if (= 3 0)
1
(* 3 (factorial (- 3 1))))
and since 3 is not 0, that expression can be replaced by
(* 3 (factorial (- 3 1)))
which is the same as
(* 3 (factorial 2))
and so that's how we write it.
Now Scheme wants to evaluate this expression (which is just another
way of writing (factorial 3), and finds that it can't proceed with the
multiplication until it figures out what (factorial 2) evaluates to.
That leads us to this substitution
(* 3 (if (= 2 0) 1 (* 2 (factorial (- 2 1)))))
which becomes
(* 3 (* 2 (factorial (- 2 1))))
which becomes
(* 3 (* 2 (factorial 1)))
We can speed things along and eliminate some of the intermediate
steps, so that the trace looks like this:
(factorial 3)
(* 3 (factorial 2))
(* 3 (* 2 (factorial 1)))
(* 3 (* 2 (* 1 (factorial 0))))
But now, computing (factorial 0) says that we can replace this
expression with the value 1. Once we no longer have user-defined
functions in the way here, Scheme can start computing all those
postponed multiplications and eventually return the expected result:
(* 3 (* 2 (* 1 1)))
(* 3 (* 2 1))
(* 3 2)
6
This perspective on how these functions work when evaluated relies
on a fundamental model of computation called "the substitution model
of evaluation". The substitution model of evaluation is a very simple
model for explaining how user-defined functions are evaluated and
return results. This model applies only in the case of purely
functional programs, like those we're writing currently. Once we move
out of the functional paradigm, we can't rely on the substitution
model any longer; because evaluation will involve keeping track of
all sorts of additional information, we'd have to use a more complicated
model called "the environment model of evaluation", but don't worry
about that for now.
The interesting thing to note here is "shape" of the growth curve of
the number of postponed computations. Each time a recursive call is
made, we add another postponed computation. And where does Scheme
remember the computations that need to be done? Well, like all
sophisticated programming language systems, Scheme tracks its
unfinished procedure or function calls...its postponed
computations...on something called an activation stack.
So what exactly is an activation stack? Read on.
III. The activation stack
So now the question becomes one of "How does your computer actually
make this recursion thing work?" It's kind of a tedious process, but
all students of computing, even short-term students of computing,
should be familiar with the mechanism, even if they've sworn never to
use recursion forever, because it's the same mechanism that allows
computers to keep track of all procedure calls, even if they're not
self-referential. And in the case of students who do use recursion a
lot (for example, 1321 students), it's a good way to study how recursion
is actually possible.
Let's say you've created a file with all the Scheme code for
evaluating the factorial function from above. You're ready to compute
stuff. If you type (factorial 3) at the Scheme evaluator, here's what
will happen:
> (factorial 3)
6
>
That looks simple, but what really happened in the computer to
compute that 6 was a boatload of stuff. You don't need to know all
the gory details, because that would turn your young minds to jelly,
so what follows is sort of a high-level description of how Scheme
(and many other languages) evaluates functions.
First, you need an introduction to a data structure called a stack. A
data structure is a means of storing and organizing data or
information. We think about data structures usually in a logical
sense...that is, we think of them abstractly in terms of what they
do, not in terms of how they're physically implemented on the
computer or some other processing platform. So a data structure could
be implemented via computer software in a number of different ways,
or it could be implemented as a special piece of hardware. For now,
we don't care...we'll look at our data structures most of the time as
an abstraction of reality...we'll ignore the details to the extent we
can. This notion of abstraction is essential to computer science, as
noted previously, and we'll revisit the concept frequently.
The stack data structure is nothing more than a linear ordered
collection of items that is special in the following way: you can
only add things to this collection (i.e., make the stack bigger) at
one end of the collection. That end is called the top of the stack,
and when you add something to the stack it's referred to as pushing
the item on the stack. To make the stack smaller, you are constrained
to take things from the ordered collection at the same end--the top.
Taking an item off the stack is called popping the stack. Another way
of looking at how this structure works is to note that the last thing
pushed on the stack will always be the first thing popped off the
stack--Last In, First Out, or LIFO as computer geeks like to say. The
classic example of a real-world stack is a spring-loaded stack of
dishes in a cafeteria, but you just don't see those things much
anymore.
Since things come off stacks in the reverse order that they went on,
stacks are good for keeping track of history. For example, if you
were working on some sort of problem and you got interrupted, you
could make a note of where you were in that problem, and put that
note on your desktop. Then you start working on whatever interrupted
you, but you're interrupted now by something else. So you write a
note about what you were doing, stick that on top of the first note,
and then deal with this latest interruption. Once you've dealt with
it, how do you pick up where you left off? Simple...take the top note
off your stack and finish whatever you were working on before you
were interrupted the second time. And when you're done with that, you
can pick up the note that's now on the top of your stack and finish
whatever you were doing before you were interrupted the first time.
Computers use this same principle to keep track of where they are in
the execution of a program. Within any function, for example, may be
calls to other functions. The system that's evaluating a given
function can't finish the evaluation until the embedded or nested
function calls are evaluated. So somehow the system must stop
evaluating the function that it's working on, remember where it was,
then evaluate the nested function call, obtain the value, then pick
up where it left off. To keep track of all these postponed
obligations (stacks are really really good for tracking postponed
obligations of all kinds), the system uses a special stack called an
"activation stack".
When you're playing with the Dr. Scheme evaluation window, and you
type something at the ">" prompt, you're telling Dr. Scheme to put
that expression on top of the activation stack and evaluate it. With
an activation stack, the thing that's on the top is most recently
active...in other words, it's what the system is working on now.
So when you type
> 2
Scheme puts the value 2 on top of the activation stack and evaluates
it. In Scheme, numbers evaluate to themselves, so Scheme puts the
value 2 in place of the 2, pops that 2 off the stack (leaving it
empty) and returns that 2 to you in the evaluation window. In that
way, Scheme is telling you that you can substitute the value 2 for
the value 2...remember the substitution model of evaluation?
When you type
> (+ 2 3)
Scheme puts that expression on top of the stack and evaluates it. It
finds the instructions for "+", plops in 2 and 3 where appropriate,
puts that conglomeration on top of the stack, and notes that whatever
is computed there substitutes for the thing just below it on the
stack (which is what pushed the current top of stack on the stack).
Scheme executes that code on top of the stack and gets the value 5.
It replaces the code on top of the stack with 5. Then it sees that
whatever result appears on top of the stack is supposed to replace
the function call below it, so it pops 5 off the stack,
and replaces the new top of stack with that value 5. So now 5 is the
only thing on the stack, since you are the calling function in a way,
and that value is returned to you in the evaluation window, so that
you can substitute 5 for (+ 2 3) in whatever it is you're doing.
When you type
> (square 2)
here's how things are pushed on the stack and popped off the stack
over a very brief amount of time. What you see below are four
consecutive "snapshots" of the activation stack as the factorial
process proceeds:
| | |
| | how to |
| | multiply |
(square 2) | (* 2 2) | 2 * 2 | 4
time ->
Yawn. That's what a trace of the activation stack looks like when Scheme
computes (square 2). Each thing on the stack is called an activation
frame, and every activation frame contains ALL the information
necessary to resume computation at exactly the point it was
interrupted, only now with the results of nested computations in
hand. There's really gobs more detail than this going on, but this is
what you need to know for now.
One other thing: when we hand trace these things, we don't typically
trace the behavior of predefined internal operations like + or *. We
just assume that stuff happens magically...we're usually only
interested in tracing the behavior of the functions we define. That
makes for shorter stack traces too. So the trace above would look
more like this:
| |
(square 2) | (* 2 2) | 4
IV. Tracing recursion
So let's take a closer look at how the activation stack is used to
manage recursion. In doing so, we might get a better understanding of
how any kind of computation is managed, and we might even end up
believing that recursion really works.
In the example below, as in the example above, we'll trace the
behavior of the stack by looking at little snapshots of the stack
contents. Just remember that snapshots to the left are taken before
snapshots to the right, so our timeline of stack behavior proceeds
from left to right.
When we first invoke factorial, say on the integer 3, what goes on
the activation stack is the equivalent of this (we'll use just "f"
instead of "factorial" to save space):
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
|(f 3) | | | | |
-------------------------------------------------------------
(these are pictures of the same activation stack as evaluation
proceeds over time)
When Scheme tries to evaluate (f 3), it finds the definition for the
factorial function, substitutes the argument 3 for the parameter n,
and then does what the definition says to do. Since 3 isn't equal to
0, the definition tells Scheme to evaluate the expression (* 3
(factorial 2)). That expression is now substituted for (factorial 3) on
the activation stack.
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
|(f 3) |(* 3 (f 2))| | | |
-------------------------------------------------------------
Now Scheme tries to evaluate the new expression on the stack. Scheme
has built-in instructions for *, so as we said above we won't include
that stuff in our trace, and Scheme knows how to evaluate 3, but now
it encounters another call to factorial. That call has to be
evaluated in the same way that (factorial 3) was, so (factorial 2) is
pushed on the stack:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | |(f 2) | | |
|(f 3) |(* 3 (f 2))|(* 3 (f 2))| | |
-------------------------------------------------------------
Evaluating (factorial 2) tells Scheme that it should really be
looking at the expression (* 2 (factorial 1)), and so that substitution
is made:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | |(f 2) |(* 2 (f 1))| |
|(f 3) |(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))| |
-------------------------------------------------------------
Looking at the new top of the stack, Scheme knows about
multiplication and the value 2, so once again we won't trace those
for the sake of brevity. Scheme deals with (factorial 1) in the same
way it handled (factorial 2), so the (factorial 1) expression is
pushed on the activation stack:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | |(f 1) |
| | |(f 2) |(* 2 (f 1))|(* 2 (f 1))|
|(f 3) |(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))|
-------------------------------------------------------------
Evaluating (factorial 1) puts the expression (* 1 (factorial 0)) on
the stack in place of (factorial 1), and evaluating that pushes
the expression (factorial 0) on the stack:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| |(f 0) | | | |
|(* 1 (f 0))|(* 1 (f 0))| | | |
|(* 2 (f 1))|(* 2 (f 1))| | | |
|(* 3 (f 2))|(* 3 (f 2))| | | |
-------------------------------------------------------------
When Scheme evaluates (factorial 0), the function definition says to
return the value 1 in place of the function call itself. So
(factorial 0) is replaced by the value 0 on the top of the stack:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| |(f 0) |1 | | |
|(* 1 (f 0))|(* 1 (f 0))|(* 1 (f 0))| | |
|(* 2 (f 1))|(* 2 (f 1))|(* 2 (f 1))| | |
|(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))| | |
-------------------------------------------------------------
One of the things that's stored in an activation frame is some
sense of how it relates to the frame beneath it. So in this case,
Scheme knows that the 1 on top of the stack can be substituted for
the (factorial 0) in the frame below. The 1 is popped off the stack
and substituted for (factorial 0) in the (* 1 ... ) expression:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| |(f 0) |1 | | |
|(* 1 (f 0))|(* 1 (f 0))|(* 1 (f 0))|(* 1 1) | |
|(* 2 (f 1))|(* 2 (f 1))|(* 2 (f 1))|(* 2 (f 1))| |
|(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))| |
-------------------------------------------------------------
Scheme now resumes evaluating the (* 1 ... ) expression, but this
time it looks like (* 1 1) instead of (* 1 (factorial 0)). (* 1 1)
evaluates to 1:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| |(f 0) |1 | | |
|(* 1 (f 0))|(* 1 (f 0))|(* 1 (f 0))|(* 1 1) |1 |
|(* 2 (f 1))|(* 2 (f 1))|(* 2 (f 1))|(* 2 (f 1))|(* 2 (f 1))|
|(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))|(* 3 (f 2))|
-------------------------------------------------------------
That value 1 on top of the stack is linked to the (factorial 1)
expression below it, and so the 1 is popped off the stack and
replaces that (factorial 1):
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
|(* 2 1) | | | | |
|(* 3 (f 2))| | | | |
-------------------------------------------------------------
(* 2 1) evaluates to 2, so the value 2 takes the place of (*2 1) on
top of the stack, and that in turn is linked to the (factorial 2)
expression in the frame below, and so on. The recursion "unwinds"
and continues on like this until the final result, 6, is the top of
the stack and the only thing on the stack:
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
| | | | | |
|(* 2 1) |2 | | | |
|(* 3 (f 2))|(* 3 (f 2))|(* 3 2) |6 | |
-------------------------------------------------------------
The value 6 is then popped off the stack and returned to who or
whatever put the original request for (factorial 3) on the stack, and
this example is done.
But I should warn you again that this is a relatively brief,
hand-wavy, and imprecise description of how the activation stack
really works. I haven't shown you all the frames that would have been
pushed on the activation stack, and I sure haven't showed you all the
information that would be in a given frame. So while this little
exercise may help you see how recursion works, or later on you may
employ a similar sort of stack trace to debug your own program, there
are lots of details you haven't seen yet... the kinds of details
you'll see if you take subsequent computer science courses.
V. Some simple recursion examples
Now that we have some idea as to how a computer can make sense out of
a function that's defined in terms of itself, it's time to look at
some other examples of simple recursion to help reinforce the
concepts that have just been learned. One of the classic exercises
in teaching recursion asks you to pretend that multiplication no
longer works on your computer and to implement your own
multiplication function using only repeated additions. (It may sound
funny, but actually, in mathematics, that's exactly how
multiplication is defined: the recursive application of addition.
Even more interestingly, if you think math is interesting, recursion
is what's used to define all the numbers...you start with 0 and 1,
and all other numbers are defined by recursively applying the
function that adds 1 to an argument, starting with the base case
argument of 0. What's 5? Why that's just 4 + 1. What's 4? That's just
3 + 1. And so on.)
One way to look at multiplication by repeated addition is like this:
a * b = a + a + ... + a where there are b terms in the right side of
the equation
But a more recursive view of this same equation looks like this:
a * b = a + (a * (b - 1))
a * 0 = 0
(This of course assumes that we're working with non-negative integers
again, just like with factorial.)
In Scheme, with a chunk of pseudocode thrown in for starters, the
corresponding function might start out looking like this:
(define (multiply a b)
(if b = 0 then return 0
else return a + (multiply a by (b - 1))))
All that stuff after the first line is just my English-like design
notes. What would real Scheme code look like? That's easy:
(define (multiply a b)
(if (= b 0)
0
(+ a (multiply a (- b 1)))))
Can you do that translation on your own? If so, great! You're
making good progress. If not, then you may want to recall these
simple rules:
1. Function calls always start with a left parenthesis, followed
by the name of the function being called, followed by any arguments
being passed to the function, followed by a right parenthesis. So
this pseudolanguage description:
(define (multiply a b)
(if b = 0 then return 0
else return a + (multiply a by (b - 1))))
would turn into this:
(define (multiply a b)
(if (= b 0) then return 0
else return (+ a (multiply a (- b 1)))))
2. You don't need to say "then return" or "else return" or even
"return" to return a value to the calling function...you just make
sure that the value to be returned is the last thing the function
evaluates. So, this:
(define (multiply a b)
(if (= b 0) then return 0
else return (+ a (multiply a (- b 1)))))
now becomes this:
(define (multiply a b)
(if (= b 0)
0
(+ a (multiply a (- b 1)))))
Occasionally students will say that we could save an operation or
two if we stopped when b = 1 instead of b = 0:
(define (multiply a b)
(if (= b 1)
a
(+ a (multiply a (- b 1)))))
Yes, we could in fact save that thing at the end where we add 0 to
the total, but note that in our rush to optimize, we've lost sight of
the original problem. Our Scheme implementation no longer matches the
original mathematical definition that we were trying to bring to
life, and now it doesn't work in a case where it used to. Try
multiplying 3 times 0 (with 0 as the b parameter) and see what
happens. It worked with the first version, but it sure doesn't work
with the second one. So don't worry about these little optimizations
that you think you see...that's not the sort of thing we're striving
for in this class. Let's all worry about getting a solution that's
well designed, easy to understand, and that works, and we'll save the
optimizations for later. As this example points out, the rush to make
sure that the computer doesn't work any harder than it absolutely has
to makes programmers, even experienced programmers, introduce
mistakes. In the effort to make things better, premature optimization
often makes things worse.
Going back to the first version of multiply, if we wanted to follow
the recursive behavior of this function when it's evaluated using the
arguments 2 and 3, we could explain it sort of like this (just like
we looked at the behavior of factorial earlier):
(multiply 2 3)
(+ 2 (multiply 2 2))
(+ 2 (+ 2 (multiply 2 1)))
(+ 2 (+ 2 (+ 2 (multiply 2 0))))
(+ 2 (+ 2 (+ 2 0)))
(+ 2 (+ 2 2))
(+ 2 4)
6
Let's do a slightly more complicated example. The first n terms of
the harmonic series are defined as
1 + 1/2 + 1/3 + 1/4 + 1/5 + ... + 1/n
Once again, however, we can rewrite this definition recursively, so
that the first n terms of the harmonic series can be defined as:
harmonic (n) = 1 if n = 1
= 1/n + harmonic (n - 1) if n > 1
(As usual, we're assuming that the arguments to the function are
valid and we're not doing any error-checking here.) So a first cut at
a harmonic function in Scheme might look like this:
(define (harmonic n)
(if n = 1 then return 1
else return 1/n + (harmonic (n - 1))))
How's that English gonna translate into Scheme? It's sooooo easy...
(define (harmonic n)
(if (= n 1)
1
(+ (/ 1 n) (harmonic (- n 1)))))
All we're doing again is converting the English into Scheme by
moving the operators (the function names) into the prefix notation
position and wrapping parentheses at the right places. And we're also
keeping in mind how we return values. It's almost mechanical at this
point. If at this point you're still having difficulty with these
examples of recursion, it is absolutely essential that you meet
with your TA and go over lots of examples of recursion. Remember that the
"if" is what makes computing useful, because it is what allows choice, and c
hoice is what makes repetition possible (how can the computer know when
to stop or when to continue otherwise?). And without repetition, the real
power of computing is lost. Recursion is the only kind of repetition you're
going to be using for awhile, and that's why upcoming homeworks provide
you with so many opportunities to practice your recursion skills.
The whole rest of the course assumes that you understand and are
comfortable with recursion and builds heavily upon that assumption.
Your exams will test your proficiency with recursion. So the time to
master this stuff is now, not the night before the homework is due,
or the night before your first exam, or the night before your final
(much too late).
OK, I'll get down off my soapbox. Let's go back to the currecnt
example. Examining the recursive behavior of the harmonic function
yields this:
(harmonic 3)
(+ (/ 1 3) (harmonic 2))
(+ 1/3 (harmonic 2))
(+ 1/3 (+ (/ 1 2) (harmonic 1)))
(+ 1/3 (+ 1/2 (harmonic 1)))
(+ 1/3 (+ 1/2 1))
(+ 1/3 3/2)
11/6
That's slightly more complicated than what we've seen so far, but not
very much so.
And if you'd like to see a decimal fraction instead of that integer
fraction, just tweak your harmonic function like this
(define (harmonic n)
(if (= n 1)
1
(+ (/ 1.0 n) (harmonic (- n 1)))))
or this
(define (harmonic n)
(if (= n 1)
1.0
(+ (/ 1 n) (harmonic (- n 1)))))
You'll get this result for (harmonic 3):
1.8333333333333333
instead of this
11/6
Why? When Scheme is doing arithmetic with all integers, it will
assume you want the result in integer form. If you combine integers
and real numbers, then it will give results in real form. That all
has to do with the notion of data types, and like all sorts of things
in this class, we'll talk more about those later on.
Copyright (c) 2003 by Kurt Eiselt. All rights reserved, with
the exception of stuff that belongs to somebody else.
Last revised: August 28, 2003