Active I/O Streams for Heterogeneous High Performance Computing

Fabian E. Bustamante and Karsten Schwan

College of Computing
Georgia Institute of Technology
Atlanta, GA 30332, USA
{fabianb, schwan}@cc.gatech.edu

Abstract

We are concerned with the attainment of high performance in I/O on distributed, heterogeneous hardware. Our approach is to combine a program's data retrieval and storage actions with operations executed on the resulting active I/O streams. Performance improvements are attained by exploitation of information about these operations and by runtime changes to their behavior and placement. In this fashion, active I/O can adjust to static system properties derived from the heterogeneous nature of distributed CPU, storage, and network devices, and it can respond to dynamic changes in system's conditions, thereby reducing the total bandwidth needs and/or the end-to-end latencies of I/O actions. Our prototype of an active I/O system, called Adios, implements I/O as a directed network comprised of streams originating at sources, destined for sinks, and routed through a number of intermediate vertices that act on the data units traversing the stream. Adaptive resource allocation methods based on this model are under development, with the goal of improving I/O performance of complex parallel programs running in shared heterogeneous computing environments.

For the full paper