I. Climbing back into the tree
Back a couple of weeks ago while I was hanging out at Piedmont Hospital
for a few days, Bryan Kennedy introduced you to the concept of tree
recursion. Think back to that lecture, and then now lets say you wanted
to add ALL the numbers in a list that looked like
this:
'(1 2 (3 4) 5 6)
What would that function look like? Well, we could use a function that
added up all the top-level elements in a list as a starting point:
(define (sum list-of-numbers)
(cond [(null? list-of-numbers) 0]
[else (+ (car list-of-numbers)
(sum (cdr list-of-numbers)))]))
That works so long as every successive car of list-of-numbers is a number.
But what do you have to do if you encounter a car or first element that
itself is a list of numbers? You have to take that big leap of faith
and assume that you have a function that when given that list of
numbers will return the sum of all those numbers. You pass that
first element to that function (which coincidentally happens to be
the function that you're writing) and add the value that it returns
to the result obtained by passing the list of the remaining elements
(the cdr) to the function that you're writing that adds up a list
of numbers. OK, that description is a little bit confusing when it's
written out like that. Here's what the function looks like. We'll
change the name a little just to make clear the distinction between
our function above and the new one. Other than that, we've already
done most of the work. We just have to add one more cond clause:
(define (sum-all list-of-numbers)
(cond [(null? list-of-numbers) 0]
[(list? (car list-of-numbers))
(+ (sum-all (car list-of-numbers))
(sum-all (cdr list-of-numbers)))]
[else (+ (car list-of-numbers)
(sum-all (cdr list-of-numbers)))]))
Depending on what you like to test for, you could write the function
differently:
(define (sum-all list-of-numbers)
(cond [(null? list-of-numbers) 0]
[(number? (car list-of-numbers))
(+ (car list-of-numbers)
(sum-all (cdr list-of-numbers)))]
[else (+ (sum-all (car list-of-numbers))
(sum-all (cdr list-of-numbers)))]))
Either way, it does the same thing. Is there some pattern from above
that might have helped? Sure. We might have started with this:
(define (name inlist)
(cond [...some test on inlist, usually(null? inlist)...
...some base value...]
[...some test on (car inlist)...
...some op on (car inlist)...
...recurse on (cdr inlist)... ]
[else ...some other op on (car inlist)...
...recurse on (cdr inlist)... ]))
Then we would just add the recursion on the car of the inlist as well as the
recursion on the cdr of the inlist that already exists.
What if we wanted to change the requirements and just count all the items
instead of finding the sum? It's the same basic function with teeny
weeny changes:
(define (count-all list-of-numbers)
(cond [(null? list-of-numbers) 0]
[(list? (car list-of-numbers))
(+ (count-all (car list-of-numbers))
(count-all (cdr list-of-numbers)))]
[else (+ 1
(count-all (cdr list-of-numbers)))]))
What if we want to make a copy of '(1 2 (3 4) 5 6) or '(a b (c (d) e) f)??
(define (copy-everything inlist)
(cond [(null? inlist) ()]
[(list? (car inlist))
(cons (copy-everything (car inlist))
(copy-everything (cdr inlist)))]
[else (cons (car inlist)
(copy-everything (cdr inlist)))]))
Just as sum-all was derived from sum, copy-everything is derived
from a previous definition of make-copy.
II. Flatten
The next function we'll construct isn't already defined in Scheme, but most
good Scheme hackers have one of these sitting around in their personal library
of Scheme functions, or they can at least reconstruct it on demand. I call
the function "flatten", and its purpose is to take any list, nested
arbitrarily deep, and return a simple list of all the atoms in the original
list. In other words, it takes a hierarchical list structure and returns a
linear list structure, with the order of the atoms intact. Or, in still other
words, it removes all the parentheses except the outermost ones:
> (flatten '(a (b (c) d (e) f)))
(a b c d e f)
>
How the heck are you supposed to make that happen? Well, the trick is to use
abstraction to break the problem into three smaller problems: flattening the
first element of the list (which could be an symbol or a list) into a linear
list, flattening the rest of the list into a linear list, and then combining
the results into a single linear list. We already know that the way we join
two lists into a single list is by using "append", so that problem is solved,
and the other two problems are handled with recursive calls to "flatten", so
the whole thing turns out to be pretty easy. Maybe the hardest part is
figuring out what the termination conditions are. Here's one way to do it:
(define (flatten mylist)
(cond ((null? mylist) ())
((list? mylist) (append (flatten (car mylist))
(flatten (cdr mylist))))
(else (list mylist))))
The first test condition was pretty straightforward. If "flatten" is passed
an empty list, it just returns the empty list and terminates. There's really
not anything else to do, and that empty list will just be "append"ed onto
other lists, resulting in no change whatsoever. In the second test condition,
we test to see if we're looking at a not-empty list. If so, then we can assume
the list has a car and a cdr, and we assume on faith that we'll eventually get
flatten to work right, so we pass the car of the list to flatten, we pass the
cdr of the list to flatten, and again assuming on faith that flatten always
returns a flattened list, we append the two results together to get one big
flattened list. If the list we're looking at is not a list of either the empty
or non-empty kind, then we assume that we're looking at something like a
symbol. What we do with that is not so obvious at first, but assuming we
know we need to include that somehow in the result that flatten returns. Where
would that happen? Probably in the line that says "append the results of the
flatten of the car and the flatten of the cdr". So if this thing that's not a
list is supposed to be appended to something, it needs to be a list too. So
if we just make a list out of this thing (by calling list or consing this
thing to ()), we make it easily "appendable".
There are other slightly different approaches to this problem, and this
isn't exactly the approach that we came up with in class. This particular
approach has the quirk that it will flatten things that aren't lists
too. For example:
> (flatten 'a)
(a)
I didn't specify what should happen in this case, so whatever your version
does is ok with me. I don't know if the result above is a bug or a feature.
It doesn't seem nearly as difficult now, does it? Once you start to believe
that recursion works without a whole lot of attention to it, then the
solutions to these little problems become a whole lot more intuitive. But
again, it takes practice... lots and lots of practice. And that's why your
homeworks are soooooo big.
III. The essence of programming greatness
The key to greatness as a computer programmer is not mastery of a
specific language, computer, operating system, or whatever, although
those sorts of things don't hurt. The best programmers all share the
ability to analyze a problem, decompose it into successively smaller
sub-problems, and then to produce simple, small, understandable
solutions to the subproblems that can be put together to create a
nice solution to the larger original problem.
This sort of approach to creating software solutions is not easy to
teach, but it can be learned through exposure to lots of examples. At
its heart is the notion of abstraction. Abstraction is the fine art
of carefully postponing or throwing away detail. We'll try to provide
you with examples of abstraction over and over in class. Your textbook
contains similar examples. So pay attention when we're designing
solutions. Abstraction is one of the fundamental concepts of computer
science, and we'll talk about it again and again.
This is one place where the students in mainstream 1321 with no
previous programming experience may have an advantage over folks like
you who have some previous experience. Much of your previous experience
has been gained through lots of late night hobby hacking, where the work
product is going to be seen only by the person writing the program.
Hobby hackers tend to ignore issues of decomposition, abstraction,
readability, and the like, and since nobody else ever sees the
programs, it's not a big deal. But out here in the real world,
programs are as much a form of communication between people as they
are between man and machine. So if you're an experienced programmer
but you don't take the time to write programs that other people can
understand, here's where you'll start. It's time to throw away those
old ways of doing things.
And no matter what your background, always keep in mind that we'd prefer
to see well-designed, easily understood programs that don't work over
poorly-designed hacked-together programs that do work. We can always
fix the former, but if anything goes wrong with the latter, we might
as well throw it away and start over. And that's not just the
Scheme-specific rantings of someone suffering from parenthesis
poisoning. Take a look at these passages from a well-known
C programming text:
To create a good program you must do more than just type in code.
It is an art in which writing and programming skills blend
themselves together to form a masterpiece. True art can be
created. A well-written program not only functions correctly,
but it is simple and easy to understand. Comments allow the
programmer to include descriptive text inside the program.
When clearly written, a commented program is highly prized.
:
:
Consider two programs. One was written by a clever programmer
using all the tricks. The program contains no comments, but it
works. The other program is well commented and nicely structured,
but it doesn't work. Which program is more useful? In the long
run, the broken one. It can be fixed. Although the clever
program works now, sooner or later all programs have to be
modified. The worst thing that you will ever have to do is
modify a cleverly written program.
From "Practical C Programming: Third Edition" by Steve Oualline,
1997, O'Reilly and Associates.
IV. A very detailed example of the abstraction process
Consider the quadratic formula that most of you will remember from
your high school math classes (sorry for the crude ASCII
representation):
________
+ / 2
-b - \/ b - 4ac
x = _______________________
2a
This formula tells us what must be computed in finding the roots or
x-intercepts of a quadratic expression, but it doesn't exactly tell
us how, at least if we look at this from the point of view of a
computer. In order to get a computer to perform these computations
(or in other words to created a computational process that finds the
x-intercepts) we're going to have to use this formula as a
specification for the design and implementation of a program.
V. The design
How would we design a program to find the roots of a quadratic
expression, following the constraints imposed by what we already know
about math, Scheme, and the functional programming paradigm?
First, that +- operator tells us that there are two roots, and
accordingly we can turn this problem of finding the roots into two
smaller problems, each finding one root. Right there we've done
what's called an abstraction---we've pushed some detail out of the
way. We give the big problem a meaningful name, and we do the same
for the subproblems. We'll call these problems "quadratic",
"pos-root", and "neg-root". Using a little bit of English and a
little bit of mathematical notation as a design language, we can
describe what we've done like this:
quadratic (a,b,c)
is defined as pos-root (a,b,c)
combined with neg-root (a,b,c)
What we've written is not Scheme. It's pseudolanguage. A couple
of years ago, we used to teach this whole course using pseudolanguage.
There's nothing inherently wrong with pseudolanguage; it's a
perfectly useful design tool. There's no reference book on
pseudolanguage, so use whatever works for you. It's a great place to
start designing solutions to problems. Here in CS 1321X, we like
pseudolanguage. (We just don't think the entire course should be
taught using pseudolanguage, as was done several years ago.)
Now back to the problem at hand. The pos-root problem can be further
abstracted. Essentially, it's a division---the result of the
numerator divided by the denominator:
pos-root (a,b,c)
is defined as numerator (a,b,c)
divided by denominator (a)
To break down the numerator problem, we find that it's the sum of -b
and that ugly square root, which has a mathematical name: the
discriminant. Well, actually, it's the thing under the square root
symbol that's called the discriminant, and we'll make sure that's
reflected herein.
numerator (a,b,c)
is defined as negate (b)
plus sqrt-of-discriminant (a,b,c)
What about the denominator? That's easy:
denominator (a)
is defined as a
multiplied by 2
As for the sqrt-of-discriminant itself, here it is...
sqrt-of-discriminant (a,b,c)
is defined as the square root of
b multiplied by b minus
4 multiplied by a multiplied by c
What about neg-root? It's pretty much the same as pos-root, except
that I'm going to have to change numerator so that it subtracts
instead of adds. For consistency, I'll go back and change the name of
numerator to pos-numerator, and I'll change pos-root accordingly:
pos-root (a,b,c)
is defined as pos-numerator (a,b,c)
divided by denominator (a)
pos-numerator (a,b,c)
is defined as negate (b)
plus sqrt-of-discriminant (a,b,c)
and then I'll reuse pos-root and pos-numerator with a little bit of
adaptation to give me neg-root and neg-numerator:
neg-root (a,b,c)
is defined as neg-numerator (a,b,c)
divided by denominator (a)
neg-numerator (a,b,c)
is defined as negate (b)
minus sqrt-of-discriminant (a)
Unless we want to decompose things like minus and plus, we have
pretty much abstracted this problem as much as we can. Since in this
course we're going to be using Scheme, we can treat each one of these
abstractions as a design for a Scheme function.
V. That was the hardpart!
Say what? Yes, it's true. Ask any experienced programmer, and he or
she will confirm it: the hardest part of writing a computer program
is generating the design of the solution, as we just did. From here
on, it's just a matter of converting our design, which we wrote in
some unofficial homemade combination of math and English notation,
which we called a pseudolanguage, into the programming language of
our choice, which could be C, or C++, or Java, or in our case Scheme.
We often encounter students in these introductory classes who don't
do well and blame their misfortune on the choice of programming
language. "I don't like [name of hated language here], and I would
have done better if you would have let me program in [name of some
other language here]," they tell us. If this sounds like you, well,
if you came to Tech to learn what you already know, you came to the
wrong place. But the real problem in these cases goes deeper than
choice of language. Typically the problems are caused when the
student doesn't take the time to think about designing a solution
before implementing that solution in the chosen programming language.
That time problem is further compounded when the student waits until
the last minute to begin working on the problem, thinking it's going
to be easier than it is. That time problem rears its ugly head again
when the student doesn't take the time to learn the rules of the
given programming, rules which are learned better through repeated
practice than by just reading some lecture notes. So make sure you
put some time into this stuff, and spread the time around. Go at
it as you would if you were learning to play a musical instrument;
practice every day. You wouldn't try to cram all your practice into
a few hours right before you saw your piano teacher again; your piano
teacher can tell the difference, and so can we.
Let me put it another way:
There are no bad programming languages.
There are only bad programmers.
So how do we turn the design into Scheme instructions? What follows
is a very long and tedious explanation, and by now you've generated
enough Scheme code that this explanation will seem, well, obvious.
So if you're confident that you can turn the pseudocode above
into Scheme, then skip ahead to section VII. On the other hand,
if you don't feel so confident, or if you think that a little
review never hurt anyone, then by all means read section VI.
VI. The implementation
Since we've used a functional notation in our design, converting the
design to working Scheme code will require nothing more than some simple
cosmetic changes. We start with this:
quadratic (a,b,c)
is defined as pos-root (a,b,c)
combined with neg-root (a,b,c)
pos-root (a,b,c)
is defined as pos-numerator (a,b,c)
divided by denominator (a)
pos-numerator (a,b,c)
is defined as negate (b)
plus sqrt-of-discriminant (a,b,c)
denominator (a)
is defined as a
multiplied by 2
neg-root (a,b,c)
is defined as neg-numerator (a,b,c)
divided by denominator (a)
neg-numerator (a,b,c)
is defined as negate (b)
minus sqrt-of-discriminant (a,b,c)
sqrt-of-discriminant (a,b,c)
is defined as the square root of
b multiplied by b minus
4 multiplied by a multiplied by c
First, we'll get rid of the commas. Scheme doesn't want commas; it
regards spaces as delimiters. We only used commas in our design
because that comes from standard math notation.
quadratic (a b c)
is defined as pos-root (a b c)
combined with neg-root (a b c)
pos-root (a b c)
is defined as pos-numerator (a b c)
divided by denominator (a)
pos-numerator (a b c)
is defined as negate (b)
plus sqrt-of-discriminant (a b c)
denominator (a)
is defined as a
multiplied by 2
neg-root (a b c)
is defined as neg-numerator (a b c)
divided by denominator (a)
neg-numerator (a b c)
is defined as negate (b)
minus sqrt-of-discriminant (a b c)
sqrt-of-discriminant (a b c)
is defined as the square root of
b multiplied by b minus
4 multiplied by a multiplied by c
The syntax of Scheme dictates that all function calls begin with a
left parenthesis and end with a right parenthesis, so let's add those
details:
(quadratic (a b c)
is defined as pos-root (a b c)
combined with neg-root (a b c))
(pos-root (a b c)
is defined as pos-numerator (a b c)
divided by denominator (a))
(pos-numerator (a b c)
is defined as negate (b)
plus sqrt-of-discriminant (a b c))
(denominator (a)
is defined as a
multiplied by 2)
(neg-root (a b c)
is defined as neg-numerator (a b c)
divided by denominator (a))
(neg-numerator (a b c)
is defined as negate (b)
minus sqrt-of-discriminant (a b c))
(sqrt-of-discriminant (a b c)
is defined as the square root of
b multiplied by b minus
4 multiplied by a multiplied by c)
Note that there are two sorts of things going on in this design.
Sometimes we're "defining" functions and other times we're "invoking"
them or referring to them or, more commonly, "calling" them. There's
nothing here that you haven't seen already. The syntax for
these two sorts of things are slightly different, and the reasons for
the differences are unimportant at this time. Just remember that
every programming language has its idiosyncracies that you have to
know, and this is one of Scheme's.
To define a new function in Scheme, we use a pre-defined function
called "define", which stands for "define function". We saw this
earlier. The syntax for function definition is this:
(define (*function-name* *zero-or-more-parameters*)
*function-body*)
So now we add the word "define" in the appropriate places, and remove
our English equivalent, "is defined as":
(define quadratic (a b c)
pos-root (a b c)
combined with neg-root (a b c))
(define pos-root (a b c)
pos-numerator (a b c)
divided by denominator (a))
(define pos-numerator (a b c)
negate (b)
plus sqrt-of-discriminant (a b c))
(define denominator (a)
a
multiplied by 2)
(define neg-root (a b c)
neg-numerator (a b c)
divided by denominator (a))
(define neg-numerator (a b c)
negate (b)
minus sqrt-of-discriminant (a b c))
(define sqrt-of-discriminant (a b c)
the square root of
b multiplied by b minus
4 multiplied by a multiplied by c)
But we still need to move a parenthesis to the right places to get
the syntax right, so let's do that now:
(define (quadratic a b c)
pos-root (a b c)
combined with neg-root (a b c))
(define (pos-root a b c)
pos-numerator (a b c)
divided by denominator (a))
(define (pos-numerator a b c)
negate (b)
plus sqrt-of-discriminant (a b c))
(define (denominator a)
a
multiplied by 2)
(define (neg-root a b c)
neg-numerator (a b c)
divided by denominator (a))
(define (neg-numerator a b c)
negate (b)
minus sqrt-of-discriminant (a b c))
(define (sqrt-of-discriminant a b c)
the square root of
b multiplied by b minus
4 multiplied by a multiplied by c)
The syntax for a plain old function invocation is this:
(*function-name* *zero-or-more-arguments*)
The difference in syntax between function definition and function
invocation has to do with what things get evaluated when, and we'll
talk about that soon. (Actually, function definition using "define"
is just a special form of function invocation.) For the time being,
just remember that most of the time you'll be using the syntax
shown just above. If we apply that syntax to calls to functions we
defined ourselves, we get this:
(define (quadratic a b c)
(pos-root a b c)
combined with (neg-root a b c))
(define (pos-root a b c)
(pos-numerator a b c)
divided by (denominator a))
(define (pos-numerator a b c)
(negate b)
plus (sqrt-of-discriminant a b c))
(define (denominator a)
a
multiplied by 2)
(define (neg-root a b c)
(neg-numerator a b c)
divided by (denominator a))
(define (neg-numerator a b c)
(negate b)
minus (sqrt-of-discriminant a b c))
(define (sqrt-of-discriminant a b c)
the square root of
b multiplied by b minus
4 multiplied by a multiplied by c)
Now what? We can get rid of all those English names for the
predefined arithmetic functions, and replace them with the Scheme
equivalents. We saw those on Thursday too; here they are again:
plus is just +
minus is just -
multiplied by is just *
divided by is just /
square root is just sqrt
See, I told you it was easy. But before we do those replacements,
remember the syntax of a function invocation: the name of the
function comes first, followed by the arguments to the function. It's
that prefix notation we talked about. So we don't just replace those
arithmetic operators, we have to move them into our prefix form and
add the appropriate parentheses:
(define (quadratic a b c)
(pos-root a b c)
combined with (neg-root a b c))
(define (pos-root a b c)
(/ (pos-numerator a b c)
(denominator a)))
(define (pos-numerator a b c)
(+ (negate b)
(sqrt-of-discriminant a b c)))
(define (denominator a)
(* a 2.0))
(define (neg-root a b c)
(/ (neg-numerator a b c)
(denominator a)))
(define (neg-numerator a b c)
(- (negate b)
(sqrt-of-discriminant a b c)))
(define (sqrt-of-discriminant a b c)
(sqrt (- (* b b)
(* 4.0 a c))))
Almost done. There's no "negate" function in Scheme. We could create
one and change the function calls accordingly,
(define (negate x)
(- x))
or we could just use the "-" operator or function with one argument,
which Scheme interprets as "subtract the argument from zero":
(define (pos-numerator a b c)
(+ (- b)
(sqrt-of-discriminant a b c)))
(define (neg-numerator a b c)
(- (- b)
(sqrt-of-discriminant a b c)))
The last thing we have to deal with is that nebulous "combined with"
operation. Functions by definition in both mathematics and Scheme
return a single value, but here we want to return two values. So what
we'll do is combine the two numeric values into a single data
structure and return that. We'll use Scheme's favorite data structure,
the list. This is something new. A data structure is just a means of
organizing information. The list is just one type of data structure,
and happens to be one that Scheme works well with.
The way we get Scheme to combine a bunch of separate entities into a
single list is to pass those entities as arguments to a function
called "list". So, if we typed (list 2.0 -5.0) to our Scheme
interpreter, Scheme would return (2.0 -5.0). Packaging up our
multiple numbers into a single structure satisfies Scheme's need to
return a single value as well as our need to see what looks like
multiple values returned by a single function. Here's what it all
looks like after we've made this last alteration:
(define (quadratic a b c)
(list (pos-root a b c)
(neg-root a b c)))
(define (pos-root a b c)
(/ (pos-numerator a b c)
(denominator a)))
(define (pos-numerator a b c)
(+ (- b)
(sqrt-of-discriminant a b c)))
(define (denominator a)
(* a 2.0))
(define (neg-root a b c)
(/ (neg-numerator a b c)
(denominator a)))
(define (neg-numerator a b c)
(- (- b)
(sqrt-of-discriminant a b c)))
(define (sqrt-of-discriminant a b c)
(sqrt (- (* b b)
(* 4.0 a c))))
Cut and paste this into Dr. Scheme and try it out. It really works!
Oh, by the way, here's another clue for you all. (The walrus was
Paul. Never mind.) Those values like 4 and 2 are called literals.
That's not a placeholder for something else, it's the literal value 4,
or 2. Whenever you insert something into your program that evaluates to
itself (like all numbers do), that's a literal. Literals can be things
other than numbers, like strings of characters. Literals are useful, but
can be overused. Use literals when you're absolutely sure you'll never
have to change those literal values. If you use a lot of literals and then
find that you have to change those values, you may end up having to change
those values in lots of places. On the other hand, if you passed those same
values as parameters, you'd only have to change the value once to effect
changes throughout the program. There are other ways to handle this
same sort of thing too, but we'll get there later.
At this point, all of you should have been able to create the
quadratic solution on your own. If you can't do it, you need to be
talking to your TA immediately and get some clarification on things
you don't understand. And if you don't know if you can do it because
you didn't bother to try, rest assured that others in this class have
tried it, and whether they succeeded or failed, you have already
fallen behind them.
VII. Why did we do all that?
Here's our program from above:
(define (quadratic a b c)
(list (pos-root a b c)
(neg-root a b c))))
(define (pos-root a b c)
(/ (pos-numerator a b c)
(denominator a))))
(define (pos-numerator a b c)
(+ (- b)
(sqrt-of-discriminant a b c))))
(define (denominator a)
(* a 2)))
(define (neg-root a b c)
(/ (neg-numerator a b c)
(denominator a))))
(define (neg-numerator a b c)
(- (- b)
(sqrt-of-discriminant a b c))))
(define (sqrt-of-discriminant a b c)
(sqrt (- (* b b)
(* 4 a c))))
What you saw in gory detail above was a step-by-step illustration of
how one uses abstraction to go from a design in some off-the-cuff
high-level design language to an actual implementation in Scheme.
Initially, we started with a formula, and we designed a working
implementation of the formula from the top down by breaking off
a piece of the problem, designing a solution to that little piece,
and abstracting away the rest of that problem. Then we started over
on the stuff we abstracted away: we broke off a piece of the
remainder, designed the solution, abstracted away the rest, and so
on. In fact, if you go back and analyze the code, you'll see very
little in the way of real Scheme, and a lot of procedure names we
invented on the fly. We just treated our problem and its subproblems
as successively refined black boxes, until those black boxes mapped
directly onto primitive operations that were already defined for us.
We were aided in this process by thinking "functionally". Thinking
"functionally" here means that your procedures access only those
values that are passed to them as arguments, that they return single
values, and that they leave no side-effects---that is, a procedure
does nothing that persists after it returns its value. This approach
to programming holds some very significant benefits for the programmer:
Once you get used to it, these functions are pretty
darn easy to write
These functions are also pretty darn easy to read
These functions are also pretty darn easy to fix (debug) when
things go wrong
Strict adherence to the functional programming paradigm
allows you to prove the correctness of your functions
mathematically. That's not something we'll explore in this class,
although it's something that might be of interest to you if you
continue with studies in computing. Non-functional
approaches to programming make mathematical proof of
correctness difficult if not impossible.
If you go back and look at the Scheme code that was created, some
things should be obvious. We generated a lot of procedures, but each
one of those procedures is very small and amazingly easy to read. In
fact, I'd argue that someone who had no Scheme exposure whatsoever
could figure out pretty quickly exactly what was intended here,
even without any supporting documentation. And that's another
thing---because we've taken the time to make our function
names very descriptive, we've reduced the need for in-line
documentation (Although we haven't eliminated the need, so don't
start thinking that you don't have to document your work! Your TAs
will be talking to you about documentation.). Also, because each of
these functions are small and adhere to our functional programming
constraints, they're a snap to debug if something goes wrong. So
while one might want to argue that this approach creates "many
unnecessary procedure calls and is therefore inefficient code" (we
wouldn't want to impose on the computer, would we?) or that it
requires too many unnecessary keystrokes (your fingertips are
probably bleeding just in anticipation of all this typing), one would
also be certifiably insane if one argued that understanding and
debugging this more "efficient" (yet still functional) version:
(define (quadratic a b c)
(list (/ (+ (- b)
(sqrt (- (* b b)
(* 4 a c))))
(* 2.0 a))
(/ (- (- b)
(sqrt (- (* b b)
(* 4 a c))))
(* 2.0 a))))
was in any way easier than understanding and debugging what we
created earlier, wouldn't one? (And note that any increase in
"efficiency" here is pretty much negligible...there are still lots of
function calls happening, which do exact some expense, and a few more
aren't likely to be noticed at run time.)
So if you're the type of experienced programmer who likes to write
programs that look like that ugly monolith immediately above, don't
do that anymore. To encourage you to change your ways, let's look
at the notion of abstraction in more depth.
VIII. Abstraction
In writing any computer program, we start with some specification of
a problem to be solved by that program and a set of primitive, pre-defined
operations that can be performed on our computer...that set of primitive
operations constitutes the programming language that we'll use.
Creating a program that solves a previously-specified problem using
the pre-defined operations in our programming language involves the
repeated application of the following three concepts:
abstraction:
giving something a name; treating something complex
as if it were simpler; throwing away detail. (Or at
least postponing it.) We can reduce any complex
thing to a simpler thing in this way, and worry
about the details later. Abstraction includes the
notion of decomposition, but it's not equivalent to
decomposition.
reference:
mentioning something by name. This allows us to use
what was previously abstracted away.
synthesis:
combining two simple things to make a more complex
thing; the opposite of abstraction.
(And, of course, those primitive, previously-defined operations are
what allow you to stop abstracting.)
These three concepts are essential to controlling the complexity of
big programs, which is one of the things we will stress in this
course, because even though you might not be working with big
programs now, you might have to do so sometime in the future. You
never know.
How important is the concept of abstraction? Well, here's a quote
from one well-known and respected computer scientist who also happens
to be one of the big movers and shakers in the world of programming
languages:
"The most important concept in all of computer science is
abstraction. Computer science deals with information and
complexity. We make complexity manageable by judiciously
reducing it when and where possible.
"I regret that I cannot recall who remarked that computation
is the art of carefully throwing away information: given an
overwhelming collection of data, you reduce it to a usable
result by discarding most of its content"
Guy Steele
Going back to our quadratic example from above, we employed
abstraction, reference, and synthesis to make it all happen. In this
context, we sometimes call abstraction "procedural abstraction"
(which differentiates it from "data abstraction"---something we'll
hear more about later on...for now, just think of it all as
abstraction).
IX. Design programs for people, not for computers
When we introduced the two different solutions for our quadratic
problem, the question naturally arose, "Which one is better?" The
answer was that the first one, the one with seven different
functions, was qualitatively better than the second one, the
monolithic single-function solution. That answer clearly met with
some skepticism, so let me explain.
That second solution does have some advantages, to be sure. First of
all, you the programmer have to type fewer characters, so there may
be some savings on your fingers. And storing away that one big
function in your computer's memory may save a little bit of memory
when compared to saving those seven smaller functions. And one could
even argue that when the second solution was in execution, there
would be a savings in the number of function calls executed. There
are the same number of multiplications and divisions and additions
and so on being done, but we have added a handful of calls in the
first version that don't exist in the second version -- calls to
pos-root, pos-numerator, and so on. Function calls do have a cost of
some sort in terms of time and space in your computer -- we'll
explain more about that in the weeks to come -- so the second
solution is a little bit more efficient than the first solution. If
all these things are true, what makes the first version better?
Those of you who have been playing with computers for a number
of years may have some idea that things like program size (smaller =
better) and program speed (faster = better) are the important
attributes to look for when evaluating program quality. That was
certainly true many years ago when computers were very expensive.
Today however, computers are cheap and getting cheaper, while at the
same time programmers are getting more and more expensive. So the
big question in software development is no longer just how
to best conserve computer resources, but also how to best conserve
human resources. Or, in other words, programmer efficiency is at
least as important, if not more so, than program efficiency.
But what exactly is programmer efficiency? Is it just getting the code
written as quickly as possible? That's certainly part of it, but the
biggest expense in software development is incurred in debugging,
maintaining, and revising code. Lots of different studies have
concluded that as much as 60 to 70 percent of the cost of any large
software product is incurred after the initial implementation. And
more often than not, the people who are involved in that last
two-thirds of the software lifecycle are not the same people who were
there for the first one-third. (And even if they were the same
people, they'll forget a lot of stuff anyway.) Thus while we engage
in the art or science of software development, we should be striving
not only for ease of design and implementation, but also for
qualities like readability, debuggability (is that a word?),
maintainability, revisability (how about that one?), and reusability.
We can do this by working toward controlling how complicated our
programs are, and we'll talk a lot about controlling software
complexity in this course. Applying principles of functional
programming is one step toward controlling software complexity.
So if you think your main concern as a programmer is "how do I save
some bits of memory?" or "how do I shave some picoseconds off my
execution time?", you need to put those ideas away until you come
across some case where those issues are important. Don't write
programs for the benefit of the computer; write for the
benefit of the people who build it, test it, evaluate it, debug it,
adapt it, and learn from it, and heed this quote from a couple of
really smart guys:
"...a computer language is not just a way of getting a computer to
perform operations ... it is a novel formal medium for expressing
ideas about methodology. Thus, programs must be written for people
to read, and only incidentally for machines to execute."
Abelson and Sussman
Another really smart guy put it this way:
"The promise...of programming languages...has yet to be fulfilled.
That promise is to make plain to computers and to other programmers
the communication of the computational intentions of a programmer or
a team of programmers, throughout the long and change-plagued life of
the program. The failure of programming languages to do this is the
result of a variety of failures of some of us as researchers and the
rest of us as practitioners to take seriously the needs of people in
programming rather than the needs of the computer and the compiler
writer. To some degree, this failure can be attributed to a failure
of the design methodologies we have used to guide our design of
languages, and to a larger degree it is due to our failure to take
seriously the needs of the programmer and maintainer in caretaking
the code for a large system over its life cycle."
Richard Gabriel
The latter statement is a bit more ominous than the former, but the
message you should take away from all this is clear: more and more,
folks in computing are realizing that programs are written for the
benefit of people, not for the benefit of computers. If you don't
clue into this soon, you're gonna be one of the first ones against
the wall when the revolution comes. (That should be familiar to
Hitchhiker's Guide to the Galaxy fans.)
So, going back to our two competing solutions, I would argue that the
first solution is easier to read and understand by virtue of the use
of abstraction and lots of meaningful function names. I would also
argue that it's easier to repair or fix (called "debugging" in
computer land) should something go wrong. Why? Well, in general
because it's easier to understand. But more specifically, it's just
a whole lot easier to isolate and correct a problem when it's hidden
in a small procedure than when it's hidden in a much larger procedure.
Also, note that in the second version, some parts of the program are
repeated. If we repeat the same set of instructions throughout a
program instead of make that set of instructions a single function
that's called from various places, we create a software maintenance
problem. That is, if that set of instructions is incorrect and
repeated many times, then to fix it we have to make many repeated
corrections. If that set of instructions is incorrect but has been
made into a single function to be called many times, then we need
only make the corrections once. These same attributes mean that the
first solution would also be easier to adapt to new problems when
appropriate. These are all people-oriented issues, but again, that's
where the expense in software development is these days. If you've
learned something different, it's probably because you learned from a
source that was living with a model of computing and software
development as it was in the 1970s. It was useful model then, but
it's generally wrong now.
The key point here is this: The goal of everything you do as a
computer programmer (and whether you think you are one or not, for
the duration of this semseter you are a computer programmer) is to
get the computer to adapt to the user (and user in this context
includes you, the programmer), not vice versa. Tools are supposed to
make their users' jobs easier, not harder, and computers and their
programming languages are certainly tools. You don't want users to
become slaves to the computer; you want to make the computer do the
work. That's why the computer was invented in the first place. So
write programs for people; don't worry about making the computer work
harder.
Copyright (c) 2003 by Kurt Eiselt. All rights reserved, with
the exception of stuff that belongs to somebody else.
Last revised: October 2, 2003