User Level Byte-Stream Layer for High Performance Network Interconnects

Problem
Small messages are becoming increasingly important in cluster computing. One way to reduce send and receive overheads for messaging so that the low latency and high bandwidths required for high performance distributed/parallel computing, even for small messages, is to provide applications with direct access to the network interface without compromising protection. This shifts most of the protocol processing to user-level where it can often be specialized and better integrated into the application thus yielding higher performance.

SCI, Scalable Coherent Interface (ANSI-IEEE Std1596-1992), is a point-to-point solution for scalability of interconnecting nodes. It provides high bandwidth (1Gb each direction per link) as well as distributed shared memory for a cluster of nodes. The Systems Lab in the College of Computing has four UltraSparc Unix machines linked with Dolphin SBus-2 SCI interconnects.

You task in this project is to implement a user level stream layer on top of the SBus-2 SCI cards. The stream library should at least be able to:

• Establish and terminate a connection;
• Send and receive data through these connections, as if writing to and reading from streams.

The minimum deliverable for this project is the implementation of such a streaming library and its use with a simple test program. Time permitting, an extension of this project asks the student to write a few micro-benchmarks with which they will evaluate the performance differences between their new user-level library and the existing kernel-level library, for different message sizes and message rates.

Background

• Dolphin SCI Interconnect White Paper.
• Dolphin SBus-2 Cluster Adapter Card Overview.
• SBus-to-SCI Adapter User's Guide

• C/C++ source code for the byte-stream library and the benchmark program;
• A short report analyzing the performance of the new stream library and possibly, any problems with its use.