I. Binary search trees
There are searches that can use additional information to help guide
the search, so that the search doesn't have to be an exhaustive search.
For example, if you could organize information in a tree in such a way
that it was sorted, you could search for a desired node in that tree
relatively quickly. To illustrate this, we'll build yet another tree,
but instead of letters in the alphabet associated with each node, we'll
use numbers to make the sorting within the tree even more obvious.
In constructing this tree, we'll follow the same organizational rules
that we've seen in our not-so-pretty nested-list tree structures up
to this point:
1. Every tree is represented by a list with three elements.
2. The first or root element must be a node (a number, in this case).
3. The second and third elements represent the descendants or
children of the root node on the left and right, respectively. These
elements can be either another tree or (), indicating that there are
no descendants on that side.
And there's one other rule that's new here:
4. Given the number associated with any root node, the numbers
in the left subtree are all less than the number at the root node,
and the numbers in the right subtree are all greater than the number
at the root node. (We're assuming for the sake of simplicity that no
number appears more than once in the tree, but in reality that's
something you'd want to deal with.)
Here's an example of one such tree:
55
/ \
/ \
/ \
25 70
/ \ / \
/ \ / \
12 30 62 80
/ \ / \ \
/ \ / \ \
6 15 60 68 99
\
\
7
And here's the nested Scheme list version of the same tree:
(55 (25 (12 (6 () (7 () ())) (15 () ())) (30 () ()))
(70 (62 (60 () ()) (68 () ())) (80 () (99 () ()))))
What would this course be without some Scheme code to look at?
Nothing, that's what. So here's some Scheme code for the search we
want to do, which again we adapted from the depth-first search
program:
(define (binary-search item tree)
(cond ((done? tree) #f)
((found-item? item (get-root tree)) item)
((< item (get-root tree))
(display (get-root tree))
(newline)
(binary-search item (get-left-subtree tree)))
(else (display (get-root tree))
(newline)
(binary-search item (get-right-subtree tree)))))
(define (done? tree)
(null? tree))
(define (found-item? item root)
(equal? item root))
(define (get-root tree)
(car tree))
(define (get-left-subtree tree)
(cadr tree))
(define (get-right-subtree tree)
(caddr tree))
Here's what happens when we run this:
> (binary-search 68 '(55 (25 (12 (6 () (7 () ())) (15 () ())) (30 () ()))
(70 (62 (60 () ()) (68 () ())) (80 () (99 () ())))))
55
70
62
68
> (binary-search 85 '(55 (25 (12 (6 () (7 () ())) (15 () ())) (30 () ()))
(70 (62 (60 () ()) (68 () ())) (80 () (99 () ())))))
55
70
80
99
#f
>
Each time this binary search procedure decides to search either the
left or right subtree, on average it's eliminating half of the tree
from the possibility of being searched. So let's say you have 15
nodes in a binary tree that has one node that's the root node, and
that node has exactly two children, and each of those nodes have
exactly two children, and each of those nodes has exactly two
children. After one comparison, the binary search function reduces
the number of nodes that could be searched from 15 (2^4 - 1) to 7
(2^3 - 1). On the next comparison, the tree to be searched has only 3
nodes (2^2 - 1), and on the next comparison the tree remaining to be
searched has only 1 node (2^1 - 1). In other words, each time that
binary search eliminates part of the tree, it's reducing the number
of nodes to be searched by a power of 2. That suggests some sort of
logarithmic time complexity, at least in the average case, and sure
enough, if you do the math, you find that the average case time
complexity for binary search is O(log n), as was mentioned on Tuesday.
II. Welcome to the relational network
We've been looking at tree-like data structures a lot lately, and by
now you're probably wondering what is the big fascination with these
things. The reason we get all tingly about trees, or any sort of
hierarchical data structure, is that they're great w0ays to organize
knowledge. Traditional linear structures, like files with lots of
records, or even to some extent simple linear linked lists like those
we've been using in this class, make it difficult to represent the
wide variety of relationships which exist between entities in the
world. Trees get us a step closer to representing the complexities of
relationships between things in the real world because they get us
closer to that notion of a relational network mentioned above. Many
times when we're using computers, we're really trying to build
computational models of some aspect of the real world, and structures
like trees and networks help us make more useful models.
III. Networks in the dictionary
We see hierarchical organizations in the real world all the time. They
may not be "pure" hierarchies, but they're hierarchical in spirit at
least. It might be easier to think of these things as "networks"
instead of hierarchies. Take for example the common dictionary. At
first glance, it looks like a very linear organization of the words in
our language. But what a dictionary really specifies is a very complex
and somewhat hierarchical map of the relationships between the words in
our language. Here are some sample definitions:
dog: any of a large and varied group of domesticated animals
related to the fox, wolf, and jackal
chihuahua: any of an ancient Mexican breed of very small dog with
large, pointed ears
bird: any of a class of warm-blooded, two-legged, egg-laying
vertebrates with feathers and wings
penguin: any of an order of flight-less birds found in the Southern
Hemisphere, having webbed feet and paddle-like flippers for
swimming and diving
ostrich: a large, swift-running bird of Africa and the Near East; the
largest and most powerful of living birds; it has a long neck, long
legs, two toes on each foot, and small useless wings
canary: a small yellow songbird of the finch family, native to the
Canary Islands
Notice that these definitions all relate the thing being defined to some
larger class of things, and then goes on to try to distinguish that
thing from other similar things. Note also that as the things being
described stray further and further from what we might think of as
being norms or stereotypes, the definitions get longer and more
detailed. For example, compare the canary (a stereotypical bird) to
an ostrich (an extremely non-stereotypical bird).
When we take the time to look at the dictionary in this way, we uncover
what is essentially a bunch of pointers from one word to others.
Because I'm trying to prove a point here, I've focused on the animal
kingdom, knowing that zoologists spend a lot of time building these
"taxonomies", or hierarchies of what is related to what. But it works
for things other than animals:
chair: a piece of furniture for one person to sit
on, having a back and, usually, four legs.
And so on. You can look up more if you like. Try "couch", "sofa",
"table", "ottoman", and whatever. No matter what noun you look up, you'll
find the same sort of pattern: relate this word to a larger class of
things, then describe some features to distinguish this thing from similar
sorts of things. Of course, I've purposely avoided dealing with another
big class of words here--a class you know as "verbs". We'll save that
topic for CS 4650, the course on natural language understanding.
The dictionary writers don't always help as much as we might like,
however:
rock: a large mass of stone
stone: the hard, solid, nonmetallic mineral matter that rock is
composed of
Except for the extra bit of information that stone is mineral matter,
all we know here is that rock is made of stone, and stone is what
makes up rock. Sigh.
In any case, we can use our high-level data abstraction, the directed
graph, to make these relationships a bit more visual. For example, from
the bird definitions, we can construct the following abstraction:
vertebrate
^
| is-a
|
| has-part
| /------------- wings
| / reproduction
| /--------------- egg-laying
| / body-temp
| /----------------- warm-blooded
bird--<< no. of legs
^ ^ ^ \----------------- 2
/ | \ \ covering
is-a / | \ \--------------- feathers
/ | \ \ movement
color / | \ \------------- flight
yellow ------canary | \
size / | is-a \ is-a
small -----/ | \ movement
| ostrich---------- run
movement | \ size
swim ----------penguin \--------- big
OK, so I fudged this a little. I had to infer that the fact that birds
have wings means that they move around by flying; the dictionary writers
didn't tell us that. And I had to infer that the fact that birds were
egg-laying told us something about their reproductive processes, and so
on. But you get the idea, no?
IV. Networks in your head
These sorts of knowledge hierarchies show up elsewhere. Independent of
this dictionary organization, psychologists in the 1960s theorized that
humans organize at least some of what they know in a similar
hierarchical fashion. For example, they said, a person's knowledge of
things in the world might be organized along these lines:
all things
/ \
/ \
/ \
/ \
physical objects abstract objects
/ \ / \
/ \ / \
/ \ time thought
/ \
inanimate animate
objects objects
/ \ / \
/ \ / \
/ \ / \
inorganics plants mammals birds
/ \ / \ / \
/ \ / \ / \
rock car dog human canary ostrich
note: assume that all links point upward
This hierarchy is by no means complete, nor is it exactly what the
psychologists, Collins and Quillian, had proposed, but it's sufficient
for our purposes.
If we think of all the upward links as being relationships of the form,
"the thing below is a subtype of the thing above," then we have
something called a "type hierarchy". And in fact, we could put the
label "S" on all those upward links, to indicate that the thing below
is a "S"ubtype of the thing above. Folks in the computing world
(particularly those in the branch called artificial intelligence or AI)
don't use the "S" or "subtype" terminology very much; instead, AI folks
use the label "is-a" in hierarchies like this, as in "a canary is-a
bird". So these hierarchies can also be called "is-a hierarchies". Note
that we went ahead and used that "is-a" label in the first diagram
above.
In any case, Collins and Quillian backed up their theory with
experiments based on this premise: If people really store knowledge in
hierarchical form, then if they're asked the right questions, we should
note significant differences in the time it takes those people to
respond correctly. For example, the time to answer "yes" to "Is a
canary a bird?" should be less than the time to answer "yes" to "Is a
canary an animate object?"
The experiments did in fact generate the right numbers, and for awhile
everyone thought the question of how human memory is organized had been
answered. But other experimenters had difficulty replicating these
results, so there was some controversy about just how Collins and
Quillian obtained their results. Nevertheless, hierarchical models of
human memory are still very popular, although they are somewhat
different in their organization than the one we've just looked at.
V. Why do the arrows point up?
Well now, that's an interesting question, no? The reason, at least from
either a psychological or AI point of view, is that humans typically
are better at answering questions like "Is a dog a mammal?" than
questions like "Name all the mammals you know." In other words, people
are better at recognition than recall or retrieval. The upward arrows
in our diagrams suggest that it would be easier to start at the "dog"
node and traverse the link up to the "mammal" node to answer "Is a dog
a mammal?", than it would be to start at the "mammal" node and try to
traverse all the downward links in an effort to enumerate all the
different types of mammals.
VI. Inheritance
We can get more utility out of our hierarchies if we add important and
distinguishing properties (or features or attributes, all of which are
indicated by the links that tend to go horizontally rather than
vertically), like we did in the dictionary example:
vertebrate
^
| is-a
|
| has-part
| /------------- wings
| / reproduction
| /--------------- egg-laying
| / body-temp
| /----------------- warm-blooded
bird--<< no. of legs
^ ^ ^ \----------------- 2
/ | \ \ covering
is-a / | \ \--------------- feathers
/ | \ \ movement
color / | \ \------------- flight
yellow ------canary | \
size / | is-a \ is-a
small -----/ | \ movement
| ostrich---------- run
movement | \ size
swim ----------penguin \--------- big
If we then allow what is called "inheritance" of these features or
attributes, we get a big win. Inheritance means that one type inherits
or takes on the properties of its supertypes, assuming that there's no
information to the contrary. So, for example, we know that a canary's
primary mode of movement is by flight, even though we don't see that
explicitly represented as a property of canaries, because we can see
that a bird (the supertype of canary) moves by flight. The canary
subtype inherits the property of flight from the bird supertype. If
we didn't allow inheritance in networks like this, we'd have to attach
the property of movement by flight to every appropriate node in the
network. Not only that, but we'd have to repeat every specific property
everywhere that we wanted it in the network, and that would cost us a
humongous amount of storage space. So inheritance buys us economy of
representation, although any program that takes advantage of inheritance
is going to have to do some extra work to search around the network and
find out which properties are supposed to be inherited.
We can also make exceptions, and say that a penguin moves primarily by
swimming, even though it's a bird. We add that property explicitly at
the "penguin" node, and it overrides the default property of movement
by flight at the "bird" node. So, in an "inheritance hierarchy" such as
this, properties are only passed from supertype to subtype when there's
no explicit information to the contrary stored with the subtype.
VII. Networks and relational databases
Everything we've shown you so far has been purely tree-like in form,
but as we've said, that's clearly not necessarily going to be true. In
fact, it's much more likely that the organization in these structures
will be much more convoluted. Consider some of the relationships which
may exist in a small company that makes cough drops:
Smith options
Brothers ---------- pay plans
|\ / \
| \ / \
| \ salaried hourly
| \ / \ / \
dept.| ---- / $800/mo / $2.00/hr
| \ /
| pay / ---- /pay
| plan/ \ /plan
| / dept.\ /
| / \/
manufacturing shipping
/ \ / \
/ \ / \
/ \ / \
Arnie Brian Chuck David
Smith Smith Smith Smith
\
\ pay
\
$2.50/hr
In an ordinary database (often called a "flat-file" database) where
all the information about an employee is contained with that employee's
record, changing the pay rate for all the salaried employees for
example requires a change to every salaried employee's record. In this
database, changing every salaried pay rate from $800 per month to $825
per month requires only one change. Anyway, welcome to the exciting
world of relational databases. Relational database work is something
that evolved from artificial intelligence ideas about how to organize
knowledge (although you'll never get a relational database person to
admit this), which in turn evolved from ideas in cognitive psychology.
VIII. The whole world is one big graph
Geographically speaking, that's true. Take a look at
http://www.mapquest.com sometime and find a route between two cities in
the United States, say Atlanta and Los Angeles. Now think about how
Mapquest might find that route. Here's a hint: look at a detailed
highway map of the U.S....hmm, looks like a network of cities and towns
connected by roads, doesn't it?
And, you may have heard about the idea called "Six Degrees of
Separation"---that is, the idea that each of us is somehow connected to
everyone else on the planet by a distance of no more than six personal
or professional relationships. That this notion might possibly be true
suggests that, at least on a relationship level, we can model the
world's population as a very very large directed graph, and that we
could find a specific connection between any two people by performing
some sort of search on that graph. That very principle is embodied in
the automated version of the momentarily popular game called "Six Degrees
of Kevin Bacon" in which the premise is that everyone in the film industry
has no more than six degrees of separation between them. You can try it out
on the World Wide Web at http://www.cs.virginia.edu/oracle and while we're
at it, the Web itself is just another graph structure...from a given Web
site you can link to a bunch of others, so you choose one, and from there
you can link to a bunch of others or go back, and so on. (If you eliminate
the commercial stuff, what's left on the Web is an interconnected network of
narcissistic memorials that people erect in honor of themselves...but I
digress.)
Copyright (c) 2003 by Kurt Eiselt. All rights reserved, with
the exception of stuff that belongs to somebody else.
Last revised: October 11, 2003