GIT-CC-01-18
Greg Eisenhauer, Fabián E. Bustamante and Karsten Schwan
College of Computing
Georgia Institute of Technology
Atlanta, GA 30332, USA
{fabianb, eisen, schwan}@cc.gatech.edu
Abstract
New trends in high-performance software development such as tool- and component-based approaches have increased the need for flexible and high-performance communication systems. High-performance computing applications are being integrated with a variety of software tools to allow on-line remote data visualization, enable real-time interaction with remote sensors and instruments, or provide novel environments for human collaboration. There has also been a growing interest among high-performance researchers in component-based approaches, in an attempt to facilitate software evolution and promote software reuse. When trying to reap the well-known benefits of these approaches, the question of what communications infrastructure should be used to link the various components arises. In this context, flexibility and high-performance seem to be incompatible goals. Traditional HPC-style communication libraries, such as MPI, offer good performance, but are not intended for loosely-coupled systems. Object- and metadata-based approaches like XML offer the needed plug-and-play flexibility, but with significantly lower performance. We observe that the flexibility and baseline performance of data exchange systems are strongly determined by their wire formats, or by how they represent data for transmission in the heterogeneous environments. Upon examining the performance implications of using a number of different wire formats, we propose an alternative approach for flexible high-performance data exchange, Native Data Representation, and evaluate its current implementation in the Portable Binary I/O library.
For the full paper
