Analyzing recursion
Last time we introduced the conditional, and with it you
were all able to invent recursion by implementing our own
version of LISP's "member" function. Now let's look at another
example of recursion. In this case, the example is a classic
when it comes to educating folks about recursion. It's computing
the factorial of an integer. The factorial function is defined
in one of two ways:
n! = n * (n-1) * (n-2) * ... * 2 * 1 if n > 0
1 if n = 0
or
n! = n * (n-1)! if n > 0
1 if n = 0
The first definition suggests a traditional iterative
approach to computing the factorial, while the second one
feels much more recursive. So let's implement the second
one:
(defun factorial (n)
(cond ((= n 0) 1)
(T (* n (factorial (- n 1))))))
If we apply our substitution model of evaluation, we can
analyze what happens when we execute this program. When we
first invoke factorial, say on the integer 4, what goes on
the program stack is the equivalent of this (we'll use "fact"
instead of "factorial" to save space):
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
|(fact 4)| | | | | |
-------------------------------------------------------
Evaluating (fact 4) results in replacing (fact 4) with the
multiplication function and two arguments, the integer 4 and
a function call of (fact 3):
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| |(fact 3)| | | | |
| | 4 | | | | |
|(fact 4)| * | | | | |
-------------------------------------------------------
Again, what's on top of the stack gets evaluated and is
subsequently replaced in our substitution model of
evaluation:
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | |(fact 2)| | | |
| | | 3 | | | |
| |(fact 3)| * | | | |
| | 4 | 4 | | | |
|(fact 4)| * | * | | | |
-------------------------------------------------------
This repeated substitution continues until we get to
(fact 0), which is a termination condition evaluating to 1:
| | | | | | |
| | | | |(fact 0)| 1 |
| | | | | 1 | 1 |
| | | |(fact 1)| * | * |
| | | | 2 | 2 | 2 |
| | |(fact 2)| * | * | * |
| | | 3 | 3 | 3 | 3 |
| |(fact 3)| * | * | * | * |
| | 4 | 4 | 4 | 4 | 4 |
|(fact 4)| * | * | * | * | * |
-------------------------------------------------------
So we hit the termination condition of our recursion, and in
the process of "unwinding" this recursion we return the
values of all these stacked or postponed computations:
| | | | | | |
| 1 | | | | | |
| 1 | | | | | |
| * | 1 | | | | |
| 2 | 2 | | | | |
| * | * | 2 | | | |
| 3 | 3 | 3 | | | |
| * | * | * | 6 | | |
| 4 | 4 | 4 | 4 | | |
| * | * | * | * | 24 | |
-------------------------------------------------------
Here's a slightly different notation for the substitution
model of evaluation. Maybe it'll give you a different
perspective on what's happening. It looks sort of like the
results of using the TRACE function (which you'll learn about
in lab, if you haven't already):
(fact 4)
(* 4 (fact 3))
(* 4 (* 3 (fact 2)))
(* 4 (* 3 (* 2 (fact 1))))
(* 4 (* 3 (* 2 (* 1 (fact 0)))))
(* 4 (* 3 (* 2 (* 1 1))))
(* 4 (* 3 (* 2 1)))
(* 4 (* 3 2))
(* 4 6)
24
The interesting thing to note here is "shape" of the growth
curve of the use of the program stack. Each time the
recursive call is made, we use up another chunk of program
stack. In fact, the use of memory grows linearly with the
integer n in the call to factorial (i.e., memory use is O(n)
for this algorithm).
What we see above is not just a picture of the factorial
procedure (i.e., the code, the program, the algorithm), it's
a look at the behavior of the "process"---the procedure in
execution. They are different, and it's important to be
aware of the distinction. In this case, we see a classic
pattern of process behavior, and it even has a name. Our
recursive procedure gives rise to what's called a "linear
recursive process".
O(n) memory use is not exactly something to be proud of.
We'd really like to do better than that. "Uh?" you might be
asking, "Is this the guy who told us not to worry about
saving a cycle here or a byte there for the sake of
efficiency?" Yes, it's still me, but here we're looking at
saving much more than a few cycles or a few bytes. Here
we're questioning whether we'll be able to compute factorials
for large integers without running into limitations of
available memory, and that's a whole different problem.
We'll take a look at how to solve that problem below.
Tail Recursion
If we go back and look at the shape of the factorial process
above, we see that the real culprit in our excessive memory
use is that each call to factorial gets replaced by a
multiplication and another call to factorial. We could stop
this from happening if we could guarantee that each call to
factorial is replaced only by another call to factorial (or
something like it). Here's how it's done:
(defun factorial (n)
(factorial-iterative 1 1 n))
(defun factorial-iterative (product counter max-count)
(cond ((> counter max-count) product)
(T (factorial-iterative (* counter product)
(+ counter 1)
max-count))))
If we look at the behavior of the process that results from
this procedure, we get an entirely different shape:
| | | | |
|(fact 4)|(fact-it 1 1 4)|(fact-it 1 2 4)|(fact-it 2 3 4)| (continued...)
----------------------------------------------------------
| | | |
|(fact-it 6 4 4)|(fact-it 24 5 4)| 24 |
---------------------------------------
Or, using the other notation:
(fact 4)
(fact-it 1 1 4)
(fact-it 1 2 4)
(fact-it 2 3 4)
(fact-it 6 4 4)
(fact-it 24 5 4)
24
Now, isn't that nice? No growth in program stack space at
all, and we didn't trade away an increase in execution time
to get this either, as there are the pretty much the same
number of function calls, multiplications, and so on. So in
this case, our memory usage is O(1) instead of O(n), and that
is something to be proud of. The shape is just a horizontal
line, which is characteristic of what's called a "linear
iterative process" (note that the process can be iterative,
referring to how the shape is produced, even though the
procedure itself is defined recursively).
To accomplish this, we used a type of recursion called "tail
recursion". In LISP, we obtain tail recursion by making sure
that the step in our procedure which is the recursive step
does not have any "work" being done "outside" the recursive
function call. Compare the corresponding steps from the two
different function definitions. Here's the first version:
(T (* n (factorial (- n 1))))))
That "(* n " part is what was killing us in terms of memory
usage. This kind of recursion is called "augmenting
recursion", meaning that there's some additional computation
added on to the recursive call, and that makes the program
stack grow. In the tail recursive version, however, there's
no augmenting recursion:
(T (factorial-iterative (* counter product)
(+ counter 1)
max-count))))
All the computation is done within the arguments to the
function call, and so the substitutions are one-for-one and
don't cause any growth of the program stack.
Here's a slightly different perspective which may help you
understand the difference between these two types of
recursion. The first version of the factorial function
postponed arithmetic operations as it progressed, and to
postpone those operations, LISP had to remember what those
operations were. To remember all that stuff, LISP had to
use a chunk of memory on the program stack for each postponed
operation.
But when the tail-recursive version of factorial is running,
no arithmetic operations are being postponed. All
computations are being done as they're needed, so no
operations must be remembered, and consequently there's
no corresponding growth in usage of the program stack.
In order to make this work, though, we had to introduce the
equivalent of variables as place holders for these embedded
computations, and we did this by introducing some additional
arguments. Those arguments in this case were established
through the creation of a helping function, which initializes
the arguments being used as variables:
(defun factorial (n)
(factorial-iterative 1 1 n))
The arguments themselves take on the roles of the sorts of
things you'd expect to see in a traditional iterative
solution:
(defun factorial-iterative (product counter max-count)
...
Here, "product" is used as the place where you store your
cumulative product, and it is initialized to the
multiplicative identity (i.e., 1). "counter" is just your
index variable, and "max-count" is the limit on the number of
"iterations" you want this thing to perform.
So, it may look like iteration, but tail recursion is most
definitely a clever and successful attempt to reap some
of the resource savings of iterative solutions while maintaining
design qualities of the functional programming paradigm.
We gain the advantages of iterative solutions without the
corresponding increase in complexity. What kind of complexity?
Well, just think about this for a minute. If I'm debugging
a recursive function that's constructed within the constraints
of the functional programming paradigm (like "factorial-iterative"),
I know that those "variables" won't be changing value while I
examine the execution of that function. That's gonna make my life
easier, in the short run as well as the long run. On the other
hand, if I'm debugging the traditional, loop-based, side-effects-laden
equivalent of "factorial-iterative", I've got three variables
which *are* changing values as that procedure is running, and
that makes it harder for me to follow what's going on, and therefore
harder for me to make sure that the procedure actually does what I
want it to do.
Sometimes the tail recursive solutions are not immediately
obvious, and sometimes they just seem natural. Just take
a look at "my-member" (never mind) to see just how obvious the tail
recursive solution can be. Like many things in this course, it's not
especially hard, but it does require a different slant on
thinking about the problem, and again like many things
in this course, it gets better with practice. In any case, going
after the tail recursive solution is worthwhile, for reasons
we've shown above. Also, some LISP systems have the ability to
recognize and further optimize tail recursive functions. The
object code that is produced is actually a simple loop (which
you never see) with variables corresponding to the arguments being
passed in the recursive function call. This gives you the elegant
expressiveness of recursion with the speed of loops or
whatever your flavor of iterative control structure might be.
You get all the good stuff, and none of the bad stuff.
You might argue, as students occasionally do, that tail recursive
solutions are harder to read, and I believe that's true for
new LISP programmers. But as you gain more experience, the
tail recursive solution becomes just another programming
cliche and no longer causes any difficulty in understanding
what's going on. Remember that lots of tasks look difficult
the first time you encounter them, but over time they become
second nature. When you were real little, your primary mode
of locomotion was crawling. At some point you tried standing
up and walking, and this was undoubtedly much more difficult
than crawling at first. But you kept at it, and now you get
around by walking instead of crawling (except perhaps the
morning after one of those particularly intense frat parties).
The moral is simply that if you practice, you get better.
Students will sometimes overgeneralize the "factorial-iterative"
example and assume that all tail recursive functions require some
helping or auxiliary function. Our definition of "my-member"
didn't require any helping function though, and it's certainly
tail recursive. The helping function is only necessary if you
need to introduce "variables" as additional arguments, so don't
get carried away with the helping functions.
The myth of efficiency
"In general, recursion leads to small code and slow execution
and chews up stack space. For a small group of problems,
recursion can produce simple, elegant solutions. For a
slightly larger group of problems, it can produce simple,
elegant, hard-to-understand solutions. For most problems,
it produces massively complicated solutions--in those cases,
simple iteration is usually more understandable. Use
recursion selectively."
Steve McConnell, "Code Complete", c. 1993 by Microsoft Press
One of the big myths that is constantly perpetuated in the
religious wars about programming paradigms and programming
languages is that functional programming in general (and LISP
in particular) is to be avoided because of all this recursion
stuff. It's hard to learn, they say, and it's obviously
inefficient.
It may be true that the concept of recursion itself is
difficult to grasp, but for most folks it seems that the real
problem is that they've been trained to think about
repetitive computations in a certain way, using index
variables and loops and so on. That training interferes with
thinking about the same problems in different ways; you'll
feel that urge to fall back on old familiar habits. That
urge should diminish as you become more comfortable with the
paradigm.
But the knock on efficiency is in fact nothing more than
mythology spread by folks who never learned the whole story.
As we've just seen in these notes:
1) If I'm thinking about it instead of acting like I'm brain
dead, I can use tail recursion and get very efficient
results. Remember that, in general, programming languages
don't make inefficient programs--programmers make inefficient
programs, regardless of their choice of language or paradigm.
2) Again, efficiency of the programmer, especially on the
back end of the software life cycle, is greatly improved
by the use of recursion (and other functional programming
ideas), assuming of course that the other folks who read
your code also aren't brain dead.
Recursion templates
After exploring the factorial function in great detail, we
spent the remainder of our time together in teams of two,
working out recursive solutions to small problems. Unfortunately,
we only had time to present one of these problems, but I'll
do a couple of others below...they might provide a little bit
of insight into how to attack your homework assignment.
Each of these solutions serves as an example of a standard
recursive form. That is, while each solution uses recursion,
each is slightly different in either the way the test is
done, what the recursive call looks like, or how the results
are combined together. We can think of each of these
different forms as "recursion templates," and once you get
the hang of how they're different and how they're similar,
you'll probably find it easy to solve problems recursively by
going to the appropriate template and plugging in the right
stuff. (The templates come from "Common LISP: A Gentle
Introduction to Symbolic Computation" by David S. Touretzky.)
Augmenting recursion
The first example is called "augmenting recursion". As you know,
this refers to the fact that in the COND clause which contains the
recursive call, there's work being done outside the recursive
call itself. This work augments the recursion, and in our substitution
model of evaluation, that's what makes the program stack grow each
time the recursive call is made. That in turn gives us that linear
(or worse) recursive process shape that we've looked at before. We saw
augmenting recursion when we constructed our first factorial function.
Here's another example of augmenting recursion. It's our own
version of LISP's "length" function, which returns the number of
top-level elements of a list:
(defun my-length (my-list)
(cond ((null my-list) 0) ;; termination
(T (+ 1 (my-length (rest my-list)))))) ;; reduction and recursion
Tail recursion
Now here's the tail recursion version of the same thing. It's
pretty straightforward: we use the helper function to introduce
a counter variable, and once we do that it's easy to do the
arithmetic inside the recursive call and avoid postponing
any computation on the stack:
(defun my-length (input-list)
(my-length-helper input-list 0)) ;; introduce counter var
(defun my-length-helper (input-list counter)
(cond ((null input-list) counter) ;; termination
(T (my-length-helper
(rest input-list) (+ 1 counter))))) ;; reduction and recursion
Conditional augmenting recursion
Here's one we didn't talk about in class, but it'll be good
to familiar with the template. The problem here is to implement our
own version of LISP's "remove" function, which takes two arguments:
some possible list element, and a list. If the possible list element is
"eql" to any top-level element of the given list, then that
element is effectively removed from the list. What is
returned is the orginial list minus any elements that are
"eql" to the possible list element. For example:
? (remove '(a b c) '((a b c) (d e f)))
((A B C) (D E F))
? (remove 'a '(a b a c))
(B C)
?
Note that while 'a is "eql" to 'a, '(a b c) is NOT "eql" to
'(a b c). However, '(a b c) is "equal" to '(a b c).
So how do we implement our own version of "remove"? The
trick here is to look at each element of the list, and if
that element is "eql" to our element to be deleted, we
recursively call "remove" on the rest of the list, thereby
discarding the element to be removed. What if they're not
"eql" and we want to keep the element? Then we "cons" that
element onto the recursive call of "remove" on the rest of the list.
What we end up with is what's called conditional augmenting recursion,
a commmon way of skipping over some elements and reconstructing with
the others. This is another standard recursive template:
(defun my-remove (element my-list)
(cond ((null my-list) nil) ;; termination
((eql element (first my-list)) ;; test to remove element
(my-remove element (rest my-list))) ;; reduce and recurse
(T (cons (first my-list) ;; don't remove element
(my-remove element ;; reduce and recurse
(rest my-list))))))
We'll do more examples soon.
Copyright 1998 by Kurt Eiselt. All rights reserved.
Last revised: October 9, 1998