I. Scheme's rules of evaluation
The "substitution" process described last week continues until Scheme
arrives at a solution (which requires that all the functions you
defined are eventually replaced by functions that were already
defined in Scheme). Getting a good grasp on how all this works
requires that you have an understanding of how Scheme evaluates
expressions. In short, the Scheme function evaluation process can be
described like this:
1. Look up and retrieve the function definition. This gets done
by evaluating the expression that comes just after the left
parenthesis. If we've done things right, that evaluation will
eventually return the function definition.
2. Evaluate the arguments to the function by passing the
arguments themselves to the Scheme evaluation function--- numbers
evaluate to themselves, symbols like parameters evaluate to the
objects they're bound to, while lists (those things that start with a
left parenthesis and end with a right parenthesis) are treated as
function calls and evaluated accordingly...unless the name of a
"special form" is sitting there, like "define" or "if", which don't
actually work like functions because they don't evaluate all their
arguments.
3. Apply the function definition to the evaluated arguments
(i.e., replace the argument place- holders in the new function
definition with the corresponding evaluated arguments).
4. Substitute the result of all that for the original function
call from step 1.
5. Pass this new expression to the evaluator.
Warning: This description of the evaluation process is a little
light on details. It should suffice for now, but as time goes by,
we'll beef this description up with details about how various types
of things get evaluated. But rather than dump all the details on you
at once, we'll add things incrementally as we need them, if we need
them.
II. The "cost" of recursion
Last week we showed you a way of tracing the stack behavior of
a recursive procedure. We based this on something called "the
substitution model of evaluation." Applying this evaluation model
to our famous factorial function:
(define (factorial n)
(if (= n 0)
1
(* n (factorial (- n 1)))))
and invoking this function with the argument 4, the trace begins with
this function call:
(factorial 3)
which can be substituted by the equivalent expression
(* 3 (factorial 2))
and so on. The whole trace looks like this:
(factorial 3)
(* 3 (factorial 2))
(* 3 (* 2 (factorial 1)))
(* 3 (* 2 (* 1 (factorial 0))))
(* 3 (* 2 (* 1 1)))
(* 3 (* 2 1))
(* 3 2)
6
The interesting thing to note here is "shape" of the growth curve of the
trace of this function in action. Each time the recursive call is made,
we postpone another computation (i.e., everytime factorial calls factorial,
another multiplication operation is added to the expression). If these
postponed obligations are saved on the activation stack, then it's pretty
obvious that as the n in (factorial n) gets bigger, the amount of space
used up in the activation stack grows accordingly. In fact, the use of
memory grows linearly with the integer n in the call to factorial (we
say that memory use is on the order of n, which is sometimes abbreviated
as O(n), pronounced "O of n" or "big O of n"...more about that stuff in
future weeks).
What we see above is not just a picture of the factorial procedure (i.e.,
the code, the program, the algorithm), it's a look at the behavior of the
"process"---the procedure in execution. They are different, and it's
important to be aware of the distinction. In this case, we see a classic
pattern of process behavior, and it even has a name. Our recursive
procedure gives rise to what's called a "linear recursive process".
Using memory in proportion to the size of the number you pass to your
factorial function is not exactly something to be proud of. It's not a
big deal if you want to find (factorial 3) all the time, but when you
want to find the factorial of a billion or something like that, you'd
like to be able to do it without running out of memory. In general, while
I don't want you to worry about saving a little bit of memory or a little
bit of time, I do want you to worry about how to write programs that can
do lots of computation without running into limitations of available
memory (as in this particular case) or limitations of available time (as
in programs that can't finish execution before the Sun burns out). Here's
how to address the former problem....
III. Tail Recursion
If we go back and look at the shape of the factorial process above, we
see that the real culprit in our excessive memory use is that each call
to factorial gets replaced by a multiplication and another call to
factorial. There is, however, another way to look at computing the
factorial of 4, for example, that doesn't keep pushing postponed
multiplications on the activation stack. For example, we've encouraged
you to look at n! as being the same as n * (n-1)!, while we've discouraged
looking at n! as n * (n-1) * (n-2) * ... * 1. But there's real value to
looking at (factorial 4) as being 4 * 3 * 2 * 1 and not 4 * 3!, if you
just know what to do with that perspective.
Here's the insight you need to have: note that
4 * 3 * 2 * 1 is the same as
12 * 2 * 1 which is the same as
24 * 1 which is the same as
24
Or we could work from the other end:
4 * 3 * 2 * 1 is
4 * 3 * 2 is
4 * 6 is
24
Either way, what we need is the ability to do our multiplications as we
count up or down and both store and update that interim product as we go.
If we can do that, then we won't have to store all sorts of postponed
multiplications on the activation stack. But how can we do that? Allow me
to show you how it's done first, and then I'll explain it.
It turns out that we'll need two functions, not just one. Here they are:
(define (factorial n)
(factorial-iterative 1 1 n))
(define (factorial-iterative product counter max-count)
(if (> counter max-count)
product
(factorial-iterative (* counter product)
(+ counter 1)
max-count)))
If we look at the behavior of the process that results from this procedure,
we get an entirely different shape:
(factorial 4)
is substituted for by
(factorial-iterative 1 1 4)
which in turn is substituted for by
(factorial-iterative 1 2 4)
The whole trace, using the substitution model of evaluation, looks like this:
(factorial 4)
(factorial-iterative 1 1 4)
(factorial-iterative 1 2 4)
(factorial-iterative 2 3 4)
(factorial-iterative 6 4 4)
(factorial-iterative 24 5 4)
24
There is a simpler version of this function; sometimes the more complicated
one above is easier for some folks to understand, others like the one below.
Take your choice:
(define (factorial n)
(factorial-iterative 1 n))
(define (factorial-iterative product n)
(if (= n 0)
product
(factorial-iterative (* product n)
(- n 1))))
The trace would look like this:
(factorial 4)
(factorial-iterative 1 4)
(factorial-iterative 4 3)
(factorial-iterative 12 2)
(factorial-iterative 24 1)
(factorial-iterative 24 0)
24
Like I said, take your choice...go with the one that makes more sense to you.
Now, isn't that nice? No apparent growth in program stack space at all, and
we didn't trade away an increase in execution time to get this either, as
there are the pretty much the same number of function calls, multiplications,
and so on. So in this case, our memory usage is O(1) instead of O(n), and
that is something to be proud of. The shape is just a flat horizontal
line (if you turn your monitor on its side), which is characteristic of
what's called a "linear iterative process" (note that the process can be
iterative, like a loop, referring to how the shape is produced, even though
the procedure itself is defined recursively).
To accomplish this, we used a type of recursion called "tail recursion". In
Scheme, we obtain tail recursion by making sure that the step in our
procedure which is the recursive step does not have any "work" being done
"outside" the recursive function call. Consequently, when we look to see
what substitutes for a call to factorial-iterative, it's just another call
to factorial-iterative with some new values, but we don't add in any
postponed compuations. Compare the corresponding steps from the two
fundamentally different function definitions. Here's the first version from
previous lectures:
(* n (factorial (- n 1)))
That "(* n " part is what was killing us in terms of memory usage. This
kind of recursion is called "augmenting recursion", meaning that there's
some additional computation added on to the recursive call, and that makes
the program stack grow. (In your textbook, augmenting recursion is called
"embedded recursion".) In the tail recursive version, however, there's
no augmenting recursion:
(factorial-iterative (* counter product)
(+ counter 1)
max-count)
All the computation is done within the arguments to the function call, and
so the substitutions are one-for-one and don't cause any growth of the
activation stack.
Here's a slightly different perspective which may help you understand the
difference between these two types of recursion. The first version of the
factorial function postponed arithmetic operations as it progressed, and
to postpone those operations, Scheme had to remember what those operations
were. To remember all that stuff, Scheme had to use a chunk of memory on
the program stack for each postponed operation.
But when the tail-recursive version of factorial is running, no arithmetic
operations are being postponed. All computations are being done as they're
needed, so no operations must be remembered, and consequently there's no
corresponding growth in usage of the activation stack.
In order to make this second approach work, though, we had to introduce
the equivalent of variables or temporary storage as place holders for
these embedded computations, and we did this by introducing some additional
parameters. Those parameters in this case were established through the
creation of a helping function, which initializes the parameters being
used as variables:
(define (factorial n)
(factorial-iterative 1 1 n))
The parameters themselves take on the roles of the sorts of things you'd
expect to see in a traditional iterative solution in a language like C or
Pascal:
(define (factorial-iterative product counter max-count)
...
If you have some programming experience with traditional iteration, the
mapping between that approach and tail recursion is fairly obvious. In our
tail-recursive solution, the "product" parameter is used as the place where
you store your cumulative product, and it is initialized to the multiplicative
identity (i.e., 1). "counter" is just your index variable, and "max-count"
is the limit on the number of "iterations" you want this thing to perform.
So, it may look like ordinary iteration, but tail recursion is most
definitely a clever and successful attempt to reap some of the resource
savings of iterative solutions while maintaining design qualities of the
functional programming paradigm.
Sometimes the tail recursive solutions are not immediately obvious, and
sometimes they just seem natural. Like many things in this course, it's not
especially hard, but it does require a different slant on thinking about
the problem, and again like many things in this course, it gets better with
practice. In any case, going after the tail recursive solution is worthwhile,
for reasons we've shown above. Also, some Scheme systems have the ability to
recognize and further optimize tail recursive functions. The object code
that is produced is actually a simple loop (which you never see) with
variables corresponding to the arguments being passed in the recursive
function call. This gives you the elegant expressiveness of recursion with
the speed of loops or whatever your flavor of iterative control structure
might be. You get all the good stuff, and none of the bad stuff. (What's
the bad stuff? We'll show you in a few weeks.)
You might argue, as students occasionally do, that tail recursive solutions
are harder to read, and I believe that's true for new Scheme programmers.
But as you gain more experience, the tail recursive solution becomes just
another programming cliche and no longer causes any difficulty in
understanding what's going on. Remember that lots of tasks look difficult
the first time you encounter them, but over time they become second nature.
When you were real little, your primary mode of locomotion was crawling.
At some point you tried standing up and walking, and this was undoubtedly
much more difficult than crawling at first. But you kept at it, and now you
get around by walking instead of crawling (except perhaps the morning after
one of those particularly intense frat parties). The moral is simply that if
you practice, you get better.
Students will sometimes overgeneralize the "factorial-iterative" example
and assume that all tail recursive functions require some helping or
auxiliary function. The helping function is only necessary if you need to
introduce "variables" as additional arguments, so don't get carried away
with the helping functions.
IV. The myth of efficiency
"In general, recursion leads to small code and slow execution
and chews up stack space. For a small group of problems,
recursion can produce simple, elegant solutions. For a
slightly larger group of problems, it can produce simple,
elegant, hard-to-understand solutions. For most problems,
it produces massively complicated solutions--in those cases,
simple iteration is usually more understandable. Use
recursion selectively."
Steve McConnell, "Code Complete", c. 1993 by Microsoft Press
One of the big myths that is constantly perpetuated in the religious wars
about programming paradigms and programming languages is that functional
programming in general (and Scheme in particular) is to be avoided because
of all this recursion stuff. It's hard to learn, they say, and it's
obviously inefficient.
It may be true that the concept of recursion itself is difficult to grasp,
but for many folks who have this difficulty it seems that the real problem
is that they have previous experience with repetitive computations in a
certain way, using index variables and loops and so on. That experience
conflicts with thinking about the same problems in different ways; your
previous programming experience may make you feel that urge to fall back on
old familiar habits. That urge should diminish as you become more
comfortable with the paradigm.
But the knock on efficiency is in fact nothing more than mythology spread
by folks who never learned the whole story. As we've just seen in these notes:
1. If I'm thinking about it instead of acting like I'm brain dead, I can
use tail recursion and get very efficient results. Remember that,
in general, programming languages don't make inefficient programs--
programmers make inefficient programs, regardless of their choice
of language or paradigm.
2. Again, efficiency of the programmer, especially on the back end of the
software life cycle, is greatly improved by the use of recursion (and
other functional programming ideas), assuming of course that the other
folks who read your code also aren't brain dead.
V. Some tail-recursive examples
Let's go back and look at the tail recursion counterparts of some other
functions we've looked at in the past. Remember your own version of multiply,
where you had to implement multiplication through repeated addition? The
tail recursive version of multiply would look like this:
(define (multiply a b)
(multiply-iterative 0 a b))
(define (multiply-iterative result operand1 operand2)
(if (= operand2 0)
result
(multiply-iterative (+ result operand1)
operand1
(- operand2 1))))
If we look at the behavior of this procedure in execution on a small example,
we'll see this:
(multiply 2 3)
(multiply-iterative 0 2 3)
(multiply-iterative 2 2 2)
(multiply-iterative 4 2 1)
(mulitply-iterative 6 2 0)
6
Now let's take another look at computing the harmonic series, this time
using tail recursion. Since we're only interested in the harmonic series for
one or more terms, there's no need to count down to zero. Consequently,
this function will look a little bit different than those we've seen.
(define (harmonic n)
(harmonic-iterative 0 n))
(define (harmonic-iterative result n)
(if (= n 1)
(+ result 1)
(harmonic-iterative (+ result (/ 1 n)) (- n 1))))
And if we look at that trace (yes, one more time) we'll see this:
(harmonic 3)
(harmonic-iterative 0 3)
(harmonic-iterative 1/3 2)
(harmonic-iterative 5/6 1)
11/6
VI. Scheme's simple data types
We could write Scheme programs to play with numbers for weeks, but we'd
probably all go crazy. So let's start working with other kinds of
information...
Like most programming languages, Scheme comes with a set of predefined
"abstract data types" or ADTs. An ADT is (1) the logical data structure
itself (an abstraction, not the detailed implementation), combined with
(2) a set of operations which work on the data structure. When we talk
about and use ADTs, we do so without regard to how they're actually
implemented in our programming language on our computer. By ignoring those
implementation details, we're working with an abstraction, hence the name
"abstract data type". As we'll explain in the near future, abstraction is
the fine art of throwing away unimportant details...the hard part is
figuring out what's unimportant...or at least postponing worrying about
the unimportant details until they become important again.
You're already familiar with some of the data types in Scheme. For example,
you've seen the boolean type (#f, #t), the symbol type (foo, define, x, y),
and the number type (3, 5.67, 11/3). Assigning some piece of information to
a data type lets us know something about the attributes of the data---what
kinds of values we might see, how the values might be used, or what kinds
of operations can be performed on the data, for example. So while you can
perform addition on 2 and 3, you can't perform addition on #t and #f...those
things are of the wrong data type to be operated on by arithmetic operators.
VII. The dotted pair
One slightly more interesting data type is called the pair, or sometimes the
dotted pair. A dotted pair is a simple collection of two things whose
individual parts are accessible separately. Any such collection that consists
of multiple pieces of data that are individually accessible might be called
"compound data", and sometimes it will be called a "record". So a pair or
dotted pair is a collection of two parts or a record with two fields.
Where do dotted pairs come from? You need a special function to construct
them, and that function is called "cons" (as in "construct"). If we type
(cons 0 2)
at Dr. Scheme's evaluator, it will return
(0 . 2)
We could retrieve the individual values from that dotted pair via two
different accessor functions, which for historical reasons are called
"car" and "cdr". They're ugly names, because they don't really describe
what they do or what they return, but they're part of computer science
history.
What "car" does is return the first part of the dotted pair (or the first
field in this record that has two fields). So
(car (cons 0 2))
returns
0
Not surprisingly, "cdr" returns the second part or the second field:
(cdr (cons 0 2))
returns
2
You might be thinking to yourself, "Well, if he thinks this dotted pair
thing is substantially more interesting than numbers, maybe he should get
a life." You're right. Dotted pairs by themselves aren't all that interesting,
and I really should get a life. Scheme programmers don't use dotted pairs
directly all that often, and even though there are valid ways to use dotted
pairs, it is usually the case that when a Scheme programmer sees a dotted
pair appear in some result, it's not the result that the programmer had in
mind.
On the other hand, dotted pairs make possible the data structure that helps
to give Scheme its distinctive look and feel. That data structure is the
list, and if by the end of this semester you don't think that the list is
a fairly interesting data type, well, I guess there's no accounting for
taste, is there?
VIII. The list
Above we talked about the dotted pair, which in and of itself isn't all
that interesting (at least that's what I think). But the dotted pair does
give us the backbone for a much more interesting data structure called
the list.
An informal way of defining a list is to say that a list is an ordered set
of any number things that must begin with a "(" and end with a ")". Those
things in the list might be lists themselves, or there might not be any
things in the list at all, which describes something called the empty list.
The following are all examples of lists, according to this not-so-precise
definition. The first list is called the empty list:
()
(a)
(a b c)
(a (b) c)
(a (b (c)))
(zoe ate the short green crayon)
(define (factorial n) (if (= n 0) 1 (* n (factorial (- n 1)))))
Actually, that informal definition isn't entirely correct, because it admits
some structures that aren't considered lists. A more precise and formal
definition for a list in Scheme is this (taken from the R5RS language
reference document that is built in to your Dr. Scheme system...just
explore the help menu):
"A list can be defined recursively as either the empty list or a
[dotted] pair whose cdr is a list."
So while our list of examples of lists above is still correct, our first
definition would incorrectly admit (0 . 2) as a list. But it's not a list
by the second definition because the cdr of (0 . 2) is 2, which is not a
list. On the other hand, (0 2) is a list, because its cdr is the list (2).
How do we know that? Well, we'll explain that very soon. But anyway,
that's why the second definition is the more precise and correct definition.
Notice that the very last list in that collection of example lists above
looks a whole lot like a function definition. That's because it IS a function
definition. All function definitions in Scheme are lists. All function
invocations in Scheme are lists (and defining a function is nothing more
than a special case of invoking a function, no?). Which leads us to talk
briefly about one of Scheme's special attributes that is shared with very
few other languages: in Scheme, there is no distinction between "program"
and "data". In Scheme, you can manipulate the list that describes how a
function is defined with exactly the same operations that you'd use to
manipulate a list containing the names of all the people you owe money to.
In fact, it's not hard to write Scheme code that generates more Scheme
code...you can write programs that write other programs on the fly. It's
an advanced concept that we probably won't do anything else with this
semester. But while such programs aren't especially easy to debug, they can
be useful. Enough for now. We'll talk more about lists next week.
Copyright (c) 2003 by Kurt Eiselt. All rights reserved, with
the exception of stuff that belongs to somebody else.
Last revised: September 2, 2003