Liveboard notes.

Assignment 1
(Due Thursday, April 10, 1:30pm)

See CS2360 Fall 95: Assignment 5
See also CS2360 Fall 95: Assignment 5 update
And see CS2360 Fall 95: Coding guidelines

Readings (useful for assignment):

• For Tuesday 4/1/97 and Thursday 4/3/97: Read chapters 3, 4, and 5.
• 2360: Lecture 10: Search
• 2360: Lecture 11: State space search
• CS3361 Winter 96: Blind Search (.ps)
• CS3361 Winter 96: Informed Search (.ps)
• CS3361 Fall 95: Search

Relevant stuff in previous courses:

• CS2360 Fall 95: Lecture 9: 10/19/95, 10/24/95: Search
• CS2360 Fall 95: Lab 5: To Boldly Search Where...
• CS2360 Fall 95: Lab 5: It is like finding a needle...
• CS2360 Fall 95: Assignment 5
• CS2360 Fall 95: Assignment 5 update
• CS2360 Winter 95: Lecture 9: 2/7/95: Searching
• CS2360 Winter 95: Lecture 10: 2/9/95: State Space Searching
• CS3361 Winter 96: Blind Search (.ps)
• CS3361 Winter 96: Informed Search (.ps)
• CS3361 Fall 95: Search

POINTS OF CONFUSION

There are several issues the book does not make sufficiently clear (from my point of view):

• The distinction between searching for a solution which is a single state
  (finding the best class schedule or solving the 8 queens problem)
  versus searching for a solution which is a sequence of states
  (finding the best route to the airport or solving the 8 puzzle)
• The distinction between search tree growing algorithms
  (search shape)
  and search directions
  (forward, backward, and bidirectional)

SEARCHING FOR SINGLE STATES (Static optimization)

• In many problems, such as finding the best class schedule, the 8 queens problem, cryptarithmetic, VLSI layout, learning (training) algorithms for neural networks, and many constraint satisfaction problems a criteria is optimized while some set of constraints is obeyed.
• Efficient "search" algorithms for these types of problems are quite different from state sequence search algorithms, typically maintaining one or a set of possible solutions (typically disconnected from each other), and using
  hill climbing/gradient descent or
  random search/simulated annealing/genetic algorithms
  to refine the set of current solutions.
• The history of how you found the best state or solution doesn't matter.
• This is quite different from searching for a sequence of states from an initial state to a goal, although it is true that the entire sequence of states can be treated as a single object which is refined by static optimization algorithms.
• The book refers to these types of problems as having a path cost of zero and explores them in sections 3.7 and 4.4 and exercises 3.7 and 3.8.
• The book describes appropriate search algorithms for these types of problems under the title "Iterative Improvement Algorithms". Of the algorithms discussed in the book, depth first, best first, hill climbing searches (which use a difference between a current solution and a goal or assess the rate of local improvement with respect to different search directions (gradients for continuous problems)) and smart random searches are the most appropriate for finding good single state solutions.

SEARCHING FOR STATE SEQUENCES (Dynamic optimization)

• In problems such as finding the best route to the airport, the 8 puzzle, missionaries and cannibals, route finding, touring/traveling salesman problems, robot navigation, path planning, and assembly planning/sequencing a sequence of states is optimized while some set of constraints are obeyed.
• These problems are characterized by requiring a sequence of states in the problem solution, having an initial state, a set of goal states or a termination condition, and there are costs for the state transitions in the solution sequence.
• Search algorithms for these types of problems typically build a tree of solutions which are connected by "legal" state transitions, and use sets of actions/operators/state transitions and evaluation/value/heuristic functions to guide search.
• Efficient algorithms for this class of problems are typically specialized for finding sequences of states. Of the algorithms discussed in the book, breadth first, uniform cost, and A* are most appropriate for finding good state sequences.

THE SEARCH DIRECTION ISSUE

Independently of how a single search tree is constructed (breadth first, depth first, uniform cost, ...) there is an orthogonal choice of which trees to build.

• Forward search builds a tree from the initial state until the goal set is reached or the termination condition is satisfied.
  
• Backward search builds a tree from the goal set until the initial state is reached.
  
• Bidirectional search performs forward and backward search simultaneously until the trees meet.
  

QUESTIONS TO ASK ABOUT ALGORITHMS

There are several questions you should ask about search algorithms
(in fact, about any AI algorithm),
all addressing the issue of which problems will the algorithm work well on:

• On which types of problems is the algorithm guaranteed to find an optimal solution?
• On which types of problems is the algorithm guaranteed to find an acceptable solution?
• What is the search efficiency?
  How does it depend on the size of the problem?
  • time (CPU) cost?
  • memory cost?
• What are the best cases for algorithm search performance?
• What are the worst cases?
• How general is the algorithm?

SEARCH ALGORITHMS:

• Breadth first.
• Depth first.
• Beam search:
  • Expand best N children at each node.
  • Compromise between breadth and depth first search.
• Uniform cost:
  • Same as breadth first for equal cost state transitions.
  • Expands tree to constant cost to get from initial state to leaf (cost-so-far).
  • Think flooding.
• Best first:
  • Uses estimate of closeness to goal state (cost-to-go) to select which leaf to expand.
• A*:
  • Combines uniform cost and best first.
  • Total-cost = cost-so-far + cost-to-go.
  • Expand leaf with smallest total-cost.
  • cost-so-far is known perfectly.
  • cost-to-go is usually an estimate.
  • if cost-to-go is sometimes an overestimate A* may miss the optimal solution.
  • Life Lesson I: May miss optimal solution due to pessimism: it is better to be optimistic.
  • If estimated cost-to-go is always zero, A* becomes uniform cost.
  • Life Lesson II: On the other hand, it is better not to be wildly optimistic.
    Uniform cost is a very expensive search algorithm.

What are useful heuristics (estimates of the cost-to-go) for the 8 puzzle problem?

TWO USEFUL DATA STRUCTURES:

• "End game" databases.
• Priority queues.

"END GAME" DATABASES:

• Everytime a problem is solved, all the states on the solution path are stored with their (optimal) cost-to-go.
• Future estimates of the cost-to-go can use the true cost-to-go if the state is in the database.

PRIORITY QUEUES:

• It is often useful to maintain a priority queue instead of or as well as a search tree during search.
• The first element in the queue is the next node to be expanded.
• New nodes are placed in the queue according to their "priority".

VALUE FUNCTIONS:

• The cost-to-go estimate (or database) is very useful in guiding search during planning, and guiding action during execution.
• Other names for this concept are evaluation functions and value functions.
• Dynamic programming is an algorithm to find correct cost-to-go/evaluation/value functions.
• DP is backward breadth first search.
• The search is terminated when all states have correct values.
• A correct cost-to-go/evaluation/value function allows greedy planners (best first search) to find globally optimal solutions.
• There is a lot of current research on how to learn cost-to-go/evaluation/value functions.