ABSTRACT |
Native Data Representation: An Efficient Wire Format for High Performance Computing
GIT-CC-01-18
Greg Eisenhauer, Fabián E. Bustamante and Karsten Schwan
College of Computing
Abstract
Flexible and high-performance data exchange is becoming increasingly
important. This trend is due in part to the growing interest among
high-performance researchers in tool- and component-based approaches
to software development. In trying to reap the well-known benefits of
these approaches, the question of what communications infrastructure
should be used to link the various application components
arises. Traditional HPC-style communication libraries such as MPI
offer good performance, but are not intended for loosely-coupled
systems. Object- and metadata-based approaches like XML offer the
needed plug-and-play flexibility, but with significantly lower
performance.
We observe that the flexibility and baseline performance of data
exchange systems are strongly determined by their `wire formats', or
by how they represent data for transmission in the heterogeneous
environments. Upon examining the performance implications of using a
number of different wire formats, we propose an alternative approach
to flexible high-performance data exchange, Native Data
Representation, and evaluate its current implementation in the
Portable Binary I/O library.
Georgia Institute of Technology
Atlanta, GA 30332, USA
{fabianb, eisen, schwan, pmw}@cc.gatech.edu
Last modified: Sat Aug 4 17:00:38 EDT 2001