Falcon and Progress: Platforms for Interactive High Performance Programs

Introduction
Related Research and Systems
Falcon - Threads based monitoring and steering
PROGRESS - a steering toolkit
References

Introduction

Contemporary high performance computing targets applications that execute for days or weeks to answer scientific and engineering questions. Extensive research efforts have focussed on monitoring these long- running applications and visualizing both the application's output data as well as performance data. Given these capabilities, many researchers speculate about the potential advantages of interactive control and guidence of these applications. Tentatively, interactive control of these high-performance applications would allow end-users to modify and adjust various components of a target application during runtime. This type of control is refered to as 'interactive program steering.' Improved application performance and more useful application results serve as catalysts for interactive program steering.

Related research and systems

For a comprehensive review of related research and prototype systems, read our annotated bibliography listed in the References section.

The Falcon Monitoring and Steering System

Falcon is a system for on-line monitoring and steering of large-scale parallel programs. The purpose of such interactive steering is to improve its performance or to affect its execution behavior. The Falcon system is composed of an application-specific on-line monitoring system, an interactive steering mechanism, and a graphical display system. The Falcon system has recently been modified to work for distributed systems as well as parallel systems by use of a software datahub that routes monitoring events appropriately. Teh framework of the Falcon system, its implementation, and evaluation of the system performance are covered in dettail in papers listed below in the references section. A complex sample application -- a molecular dynamics simulation program (MD) -- is used to motivate the research as well as to evaluate the performance of the Falcon system. Also, an atmospheric modelling simulation has been evaluated using the Falcon on both parallel and distributed platforms. See the Falcon Monitoring and Steering System page for more info.

Our Solution -- a Steering Toolkit

Progress is a toolkit for developing steerable applications. Users instrument their applications with library calls and then steer parallel applications with Progress runtime system. Progress provides steerable objects which encapsulate program abstractions for monitoring and steering during program execution. Once created, steering objects are known to and manipulated by Progress' two components: (1) a server executing in the same memory space as the target program and capable of inspecting and manipulating program state, and (2) a potentially remote client providing command and graphical interfaces. Developers instrument their applications with the Progress toolkit library to create and maintain these steering objects. The server maintains information about the steering objects and performs steering actions on the application. This toolkit provides sensors, probes, actuators, function hooks, complex actions, and synchronization points. Progress' server is built on a Mach-compatible Cthreads library; it is a general toolkit for use with a variety of multithreaded, C programs executing on multiprocessors. Progress has been applied to several large-scale parallel application programs, including a molecular dynamics code and an N bodies simulation. It is currently being used with a complex global atmospheric modeling code.

We are currently integrating the Progress and Falcon systems as well as improving support for heterogeneous systems.

References