6220: High Performance Computing


    George Biros

    Class description

    This is a graduate course in parallel algorithms and high-performance computing. It is intended to be a standard part of the Computational Science and Engineering curriculum that have strong background in programming and algorithms. However, students from other research areas with an interest in parallel algorithm design are welcomed to attend. Topics include parallel non-numerical and numerical algorithms for shared and distributed memory architectures:
    • Networks, communications, load balancing, data locality, overlapping communication/computation
    • Models: Work-Depth, PRAM
    • Programming: GPUs, Message Passing Interface, OpenMP, Vectorization
    • Non-numerical algorithms: reduction, sorting, searching, pointer jumping, trees, graph partitioning
    • Numerical algorithms: linarN-body algorithms, FFT, linear algebra


    • Five in-class quizzes (40%)
    • Six homeworks (30%)
    • Semester-long project (30%)

    Recommended texts


    Ananth Grama, George Karypis, Vipin Kumar, Anshul Gupta, ``An Introduction to Parallel Computing: Design and Analysis of Algorithms'', Second Edition, 2003
    Available online

    Recommended books

    • Joseph Jaja, "Introduction to Parallel Algorithms",
    • Thomson F. Leighton, ``Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes''
    • Gropp & Snir, ``MPI: The Complete Reference''


    C/C++ programming, sequential algorithms