"LISP programmers know the value of everything but the cost of nothing."
- Alan Perlis
Append: the black hole of efficiency
One of the functions you were asked to construct as part of your last
homework was something called "my-append", your own version of LISP's
"append" function. As you recall, "append" joins two lists together
like this:
> (append '(a b) '(c d))
(A B C D)
Joining lists is a very useful thing, so "append" will no doubt
place high on your list of frequently-used functions. Young
LISPlings are prone to overusing "append" however. It's often
the case that LISP programmers want to construct lists (duh),
and they see two different ways of doing that. One way is to
build a list by "cons"ing elements to the front of some initial
list (often NIL):
(cons element initial-list)
while the other way is to build a list by "append"ing
elements to the end of the initial list:
(append initial-list (list element))
At first glance, these operations seem to do the same sorts of things,
the only difference being that they work on opposite ends of the list.
Consequently, you might infer that these two operations are roughly
equivalent in terms of how they work and the costs that they incur.
Nothing could be further from the truth.
What does "cons" do?
If you think back to your second homework assignment, you'll recall
that we didn't ask you to implement "my-cons". Why? Because you
can't. It's a fundamental operation that's built into the compiler.
If you wanted to see how it really works, you'd have to go into
the guts of the system. But at a higher level, you can think of
"cons" as working like this:
Each time the LISP system comes across a call to "cons", LISP allocated
a couple of words of free memory--something called a "cons cell". This
cons cell is then filled with the appropriate pointers to whatever
structures are being consed, and then a pointer to that cons cell
is returned. Thus the cons cell becomes the new first element
in the structure that was consed to, and the pointer that is returned
is pointing to that new first element.
You can think of a cons as being a fundamental unit of expense in
LISP programming. A cons has a measurable cost both in terms of time
to execute and amount of memory used. An extra cons here or there
is no big deal, but we'd like to make sure that we don't end up
using lots and lots of conses when we don't need to.
What does "append" do?
As noted previously, "cons" and "append" seem like different sides
of the same coin. One tacks stuff onto the front of a list, and the
other tacks stuff onto the back of a list. But take a careful
look at an implementation of append using cons:
(defun my-append (list1 list2)
(cond ((null list1) list2)
(T (cons (first list1) (my-append (rest list1) list2)))))
Looking at this version of append, you can see that appending something
onto the end of a long (or growing) list is computationally expensive.
You have to cons together a copy of the first list before you can
run a pointer from the end of that copy to the second list. You
usually have a choice as to how you build a list...you can either
build it by consing elements to the front of the empty list (and then
"reverse" the list if necessary), or you can build it by appending
elements to the back of the empty list (which often seems more
intuitively appealing). But if you're building really big lists,
it turns out that the latter approach is computationally much more
expensive than the former. So if you have a choice of building a list
by consing elements to the front and reversing the list once versus
appending elements to the end, make sure you use the cons-and-reverse
approach. It's not just a little savings in both time and cons cells,
it's a big big big (potentially exponential) savings in time and cons
cells. And it has nothing to do with whether "append" is implement with
augmenting recursion or tail recursion (that's about stack space, not
time or cons cells).
Here's an example that might help illustrate the the big differences in
efficiency between our two choices. Here's a tail-recursive version
of "remove":
(defun my-remove (item input-list)
(my-reverse (my-remove-helper item input-list nil)))
(defun my-remove-helper (item input-list result)
(cond ((null input-list) result)
((eql item (first input-list))
(my-remove-helper item (rest input-list) result))
(T (my-remove-helper item (rest input-list)
(cons (first input-list) result)))))
(defun my-reverse (revlist)
(my-reverse-helper revlist nil))
(defun my-reverse-helper (revlist result)
(cond ((null revlist) result)
(t (my-reverse-helper (rest revlist)
(cons (first revlist) result)))))
(Note that I've included "my-reverse" just so you don't think I'm
cheating with some super-optimized version of reverse.)
That version builds the new list, without the elements being removed,
by consing the list together and reversing it. Here's another approach
that builds the same list by appending things to the end of the list,
which may seem more intuitively obvious, but it turns out to be much
more expensive:
(defun my-remove-ugly (item input-list)
(my-remove-ugly-helper item input-list nil))
(defun my-remove-ugly-helper (item input-list result)
(cond ((null input-list) result)
((eql item (first input-list))
(my-remove-ugly-helper item (rest input-list) result))
(T (my-remove-ugly-helper item (rest input-list)
(append result
(list (first input-list)))))))
So let's compare the behavior of these two tail-recursive solutions
to "remove" which differ only in whether they use cons or append to
construct the result. I build a 1,000 element list that looks like
this: (a x a x ... a x) and then asked "my-remove" (the tail-
recursive version above) to remove every other element:
? (time (my-remove 'a x))
(MY-REMOVE 'A X) took 1 milliseconds (0.001 seconds) to run.
8,000 bytes of memory allocated.
Macintosh Common LISP barely noted how much time was involved.
And it only took 8,000 bytes of memory (in terms of cons cells) to build
the list and reverse it. What did the "ugly" version of the same
function do?
? (time (my-remove-ugly 'a x))
(MY-REMOVE-UGLY 'A X) took 72 milliseconds (0.072 seconds) to run.
Of that, 1 milliseconds (0.001 seconds) were spent in The Cooperative
Multitasking Experience.
1,002,000 bytes of memory allocated.
Hmmm. Noticeably worse performance. But look what happens when we
increase the length of the input list by a factor of 10 (i.e., a
10,000 element list):
? (length x)
10000
? (time (my-remove 'a x))
(MY-REMOVE 'A X) took 11 milliseconds (0.011 seconds) to run.
80,000 bytes of memory allocated.
The time increased a little bit with the more efficient version, and the
memory needs increased by a factor of 10, proportional to the increased
size of the input list. But when we run the less efficient version, watch
out:
? (time (my-remove-ugly 'a x))
(MY-REMOVE-UGLY 'A X) took 23,439 milliseconds (23.439 seconds) to run.
Of that, 747 milliseconds (0.747 seconds) were spent in The Cooperative
Multitasking Experience.
14,678 milliseconds (14.678 seconds) was spent in GC.
100,020,448 bytes of memory allocated.
Time requirements went up a lot, and LISP spent a lot of time in memory
management, trying to reclaim memory (i.e., garbage collect = GC) it
could use. And the memory requirements increased not by a factor of 10,
but by a factor of 100! And note that this dramatic difference in behavior
occurs even though we're using tail-recursive functions with the optimizing
compiler fully enabled.
The moral to this story is that simple little decisions like using
append instead of cons can cost you lots of time and lots of memory.
In this case, the difference appears to be linear growth in memory usage
versus exponential growth in memory usage. And it looks like the
differences in growth in time needed to complete the computation is
linear versus exponential too. So while we don't want you to sacrifice
readablility of your code to save a few bytes here or a few cycles there,
on the other hand we DO want you to worry about linear versus exponential
differences...these are the kinds of differences that determine whether
your program will finish in a few seconds or whether it will still be
running when the Sun burns out. Don't be a brain-dead programmer,
regardless of the language you're using or the problem you're tyring to solve.
Multiple recursion
The next function we'll construct isn't already defined in
Common LISP, but most good LISP hackers have one of these
sitting around in their personal library of LISP functions,
or they can at least reconstruct it on demand. I call the
function "flatten", and its purpose is to take any list,
nested arbitrarily deep, and return a simple list of all the
atoms in the original list. In other words, it takes a
hierarchical list structure and returns a linear list
structure, with the order of the atoms intact. Or, in still
other words, it removes all the parentheses except the
outermost ones:
? (flatten '(a (b (c) d (e) f)))
(A B C D E F)
?
How the heck are you supposed to make that happen? Well,
the trick is to use abstraction to break the problem into
three smaller problems: flattening the first element of the
list (which could be an atom or a list) into a linear list,
flattening the rest of the list into a linear list, and then
combining the results into a single linear list. We already
know that the way we join two lists into a single list is by
using "append", so that problem is solved, and the other two
problems are handled with recursive calls to "flatten", so
the whole thing turns out to be pretty easy. Maybe the hardest
part is figuring out what the termination conditions are. Here's
the version that our chalkboard victim provided in class:
(defun flatten (mylist)
(cond ((null mylist) nil)
((listp (first mylist)) (append (flatten (first mylist))
(flatten (rest mylist))))
(t (cons (first mylist) (flatten (rest mylist))))))
The first test condition was pretty straightforward. If "flatten"
is passed an empty list, it just returns the empty list and
terminates. There's really not anything else to do, and that
empty list will just be "append"ed onto other lists, resulting
in no change whatsoever. In the second test condition, we test
to see if the first element of the input list is itself a list.
If so, we flatten that list, and since the result of flattening
is also a list, we can append the result of flattening the first
element of the list to the result of flattening the rest of the
list. In the final case, if the first element proved not to be
a list, then it must be an atom. Then we just cons that atom
onto the front of the result of flattening the rest of the list.
It doesn't seem nearly as difficult now, does it? Once you
start to believe that recursion works without a whole lot
of attention to it, then the solutions to these little problems
become a whole lot more intuitive. But again, it takes practice...
lots and lots of practice. And that's why your homeworks are
soooooo big.
Here's another version that I tend to think of when I think
of flattening something:
(defun flatten (my-list)
(cond ((null my-list) nil)
((atom my-list) (list my-list))
(T (append (flatten (first my-list))
(flatten (rest my-list))))))
It's the same as the one above, except we've just reversed the
second and third conditions.
The type of recursion that's going on here has several names.
It can be called "multiple recursion" or "tree recursion", or
it can be called "car/cdr recursion", which is a special kind
of multiple or tree recursion.
"Multiple recursion" is an appropriate name because, from the
point of view of our substitution model of evaluation, each
call to "flatten" is substituted on the program stack by
multiple calls to "flatten".
"Tree recursion" is an appropriate name if we think about how
the original problem is broken down: we start by trying to
flatten the original list; that problem becomes two problems
of trying to flatten two parts of that list; each of those
problems spins off two more problems, and so on. If you were
to sit down and draw a little graph of how the original
problem was divided into two subproblems, and each of those
was further divided, and on and on, you'd get yourself a nice
little tree-like structure. (Also, note that a hierarchical
list structure can be interpreted as a tree structure, and
that "flatten" can be viewed as a preorder tree traversal
procedure. But we'll see more on that later.)
Finally, "car/cdr recursion" is appropriate in this
particular use of tree recursion because the initial problem
is broken down into recursive calls of "flatten" on the
"first" (or "car") of the list and the "rest" or ("cdr") of
the list. Today, we might be better off calling it
"first/rest" recursion, but we'd lose that sense of history
and tradition.
The important thing to remember here is that multiple or tree
recursion is a very standard way to travel around really
complex data structures, or to solve some problems with
really complex decompositions. We'll be seeing this
recursion template pop up in lots of different places.
Copyright 1998 by Kurt Eiselt. All rights reserved.
Last revised: October 19, 1998