Lectures:  TR 3:05-4:25 pm
Location:  Whitaker 1214

Instructor:  Edmond Chow
Office Hours: Mondays 3-4 pm in KACB 1312

TA: Long Tran
TA E-mail:  ltran3@gatech.edu
TA Office Hours: Thursdays 9-11 am in KACB 1305

Course Description

Introduction to the state-of-the-art iterative methods for solving linear and nonlinear systems of equations. This will be a very practical course, involving Matlab programming and a student-defined project.


Numerical Linear Algebra (CSE/MATH 6643) or equivalent. (Note that Numerical Linear Algebra is a completely different course than Linear Algebra. The latter is an undergraduate math course, sometimes taught along with differential equations, while the former is a graduate level course.) The assignments will require Matlab programming (at least at the level of CS 1371).


  • Sparse matrices and review of direct methods
  • Basic iterative methods (splitting methods, Jacobi, Gauss-Seidel, SOR)
  • Chebyshev iterative method and matrix polynomials
  • Krylov subspace methods (conjugate gradient method, GMRES, etc.)
  • Projection method framework
  • Methods based on biorthogonalization
  • Iterative methods for linear least squares
  • Preconditioning
  • Multigrid methods
  • Domain decomposition
  • Nonlinear systems of equations (fixed point methods, Newton, Broyden, Newton-Krylov and other Newton variants for large problems)
  • Line search and global convergence
  • Contraction mapping and local convergence theory
  • Nonlinear least squares (Gauss-Newton, Levenberg-Marquardt)
  • Related ideas in optimization

Learning Objectives

Students will develop facility with iterative methods for the numerical solution of linear and nonlinear systems, and their analysis. The students will be able to:

  • Given a linear or nonlinear system, choose an appropriate numerical solution method based on the properties of the system
  • Evaulate a method for its convergence and computational cost, including parallel computing aspects
  • Diagnose convergence problems of iterative solution methods
  • Select or design a method or approach for preconditioning the solution of specific problems
  • Use Matlab or other numerical software for solving systems of equations


40% Matlab mini-explorations of concepts covered in class

20% Analytical assignments

30% Student-defined project

10% Class participation

Discussion Forum

A discussion forum is open on T-square. You are encouraged to use it to post your questions, to answer other students' questions, and to discuss all things related to our course material.

Distance Learning Students

All deadlines are on a 1-week later schedule. Also, you are exempt from the class participation requirement.


  • Iterative Methods for Sparse Linear Systems, 2nd edition, by Yousef Saad, SIAM, 2003.
  • Numerical Methods for Unconstrained Optimization and Nonlinear Equations, by J. E. Dennis, Jr. and Robert B. Schabel, SIAM, 1996.
  • Matrix Computations, 4th edition, by Gene Golub and C. F. van Loan, Johns Hopkins, 2013.
You should already have the third book from your course in Numerical Linear Algebra. You can order the first two books from SIAM, here. You can get a 30 percent discount if you are a SIAM member. As a student, you can join SIAM for free, since Georgia Tech is an Academic Member. Check it out here! These two books are also available electronically from the Georgia Tech library.