Distributed and embedded systems have limited resources, often due to cost considerations. Furthermore, resource availability and user needs change continuously, driven by user behavior or external changes such as interference in wireless communications. Our group is developing flexible technologies to deal with dynamic resource changes and to exploit and adjust to runtime changes in user needs, with the goal of leveraging both in order to improve the experiences of end users. The target environments of our work include (1) wired homes in which high end data (e.g., video) transfers and computations (e.g., video analysis as needed in home security, or immersive, multi-player games) are ongoing and (2) wireless environments in which end users interact continuously, via portable and wireless devices.

The research areas being investigated address specific devices like PDAs and laptops and the wireless communication media they employ, with research topics ranging from (1) the creation of middleware that promotes the effective distributed programming of these target platforms (both using Java and also investigating higher performance middleware), to (2) extending the Linux kernel at runtime in order to dynamically configure its use in embedded and ubiquitous systems, to (3) investigating specific extensions including those of protocol stacks to improve certain applications' operation. Artifacts resulting from our work include the JECho and ECho middleware systems, the ASAN project investigating how to improve communication co-processor firmware, the ELinux project constructing an extensible Linux for embedded systems, and 'quality channels', a layer on top of Microsoft NT, ELinux , and configurable firmware; this layer exploits the configuration options offered by lower system levels, thereby shielding end users from having to understand system-level details. Finally, runtime monitoring extensions being developed for ELinux observe both communication behavior and processing behavior, so that runtime configuration can take advantage of knowledge about both.
 
 

1 Runtime Quality Management

1.1 Background -- Continuously Useful End User Services

Distributed, embedded applications that interact with human end users, with external devices, and in ubiquitous settings are highly dynamic. After all, human users come and go, devices are turned on and off, and devices are interesting to users only as long as they are in range and able to contribute to their ongoing activities. As a result, runtime changes occur in such applications' sets of active tasks and in the resource needs of such tasks. Simultaneously, there are dynamic changes in resource availability on the heterogeneous, parallel and distributed platforms on which these tasks run. For instance, wireless communications are subject to interference, to loss of bandwidth due to temporary disturbances, and to disconnection when out of range. Yet, end users continuously require certain levels of service from distributed applications, or they will consider them to be non-functional. These facts hold for a broad range of multimedia applications and in general, for interactive services supplied to homes and communities and also within such settings.

Our research efforts seek to provide continuously useful services to end users, despite dynamic changes in service needs and platform resources. Our approach to continuous service provision is to dynamically manage application behavior and therefore, resource needs, while also continuously considering resource availability. For instance, by being conscious of a human user's current actions and needs at the program/application level, certain corrective actions may be taken in response to loss of resources. One example is the resolution of multi-media data being generated, processed, and transferred, which may be reduced in order to improve execution speed, reduce bandwidth requirements, or minimize power needs when battery levels are low. Another example is the differential application of security processing. When security concerns do not currently exist, computing overheads may be reduced by elimination of encryption or authentication processing and/or reductions in key sizes. In fact, often such runtime adaptation of embedded applications is of critical importance, because without it applications simply cannot function at the levels of quality required by end users.

1.2 Heterogeneous Target Systems

The hardware used in our effort ranges from server systems, wired to PCs and to other machines via Ethernet, to wireless-connected laptops and PDAs. The software platforms utilized include the Linux and Palm operating systems, the Java, C, and C++ languages, and the innovative middleware and additional services in part produced by this work. The applications with which we are experimenting include media streaming (voice and video), distributed video games, and higher end interactive applications with significant processing and communication requirements (e.g., high definition data, image interpretation, etc.).

Figure 1 illustrates one of the hardware platforms our project utilizes and the target environments in which they are deployed.
 
 

2 Project Contributions

Our project makes several specific contributions to the area of distributed embedded systems. The remainder of this overview paper will describe each of the specific efforts and their contributions to our group's global vision. Also described are two additional efforts affiliated with our work, one being initiated at the Univ. of Georgia, and two affiliated efforts at Georgia Tech.
 

2.1 (J)ECho -- Flexible Middleware for Embedded, Heterogeneous Systems
 
 

Figure 2 illustrates the ECho approach. ECho uses binary code generation to `move' code across processor platforms. It operates across both Windows NT and Unix platforms (including Linux). In this figure, we show the use of ECho channels in a wireless, heterogeneous environment. The client dynamically installs handlers on the source by sending control sequences over an event channel.

JECho is implemented with Java. Its contributions to our project are its ability to operate on any platform supporting at least the KVM virtual Java machine, and its ability to interoperate smoothly with ECho-based communications. Such interoperations is defined as (1) the ability to automatically translate ECho to JECho communications (and vice versa), (2) the resulting ability to construct distributed embedded applications that include both native code and interpreted Java components, and (3) the ability to rapidly prototype application components with Java and subsequently, re-code selected, performance-sensitive application components with native code approaches, yet not losing valuable knowledge about application structure and functionality. JECho also provides interesting features exploiting Java's abilities, termed 'eager handlers'. These features are described in detail in the student paper associated with this work.

Figure 3. A laptop streaming data to a palmtop running JECho (additional, small events issued by the palmtop to control the data stream are not shown in this figure)
 
 

2.2 Robust Distributed Computing on Wireless Platforms

An ongoing effort is to build robust versions of ECho and JECho across wireless-connected systems, where we must deal with issues like disconnection, power-awareness, and communication security. While our middleware is already operational across wireless systems, technical issues to be addressed include topics like runtime caching of meta-information about the data being exchanged, suitable representations of such meta-information able to be shared across a wide range of target platforms (i.e., using XML), and the robustness of the distributed applications assembled using such middleware. What we mean by robustness in this context includes (1) continuous service delivery subject to end users' dynamically changing needs and to platform resource availability, (2) meeting the timing constraints imposed by specific devices and applications that use them, and (3) meeting non-performance-based constraints, such as the ability to react to low power levels, to security needs, etc.
 
 

2.3 Runtime Configuration of Open Source Embedded Systems

We are developing runtime configuration methods to be applied to open source operating systems suitable for embedded platforms, with our current work focused on the Linux OS kernel. Since our group's aim is to continuously manage user needs and the platform resources applied to these needs, we are creating new dynamically loaded code modules for the Linux operating system kernel. The purpose of these modules is to create an extensible Linux for our target embedded platforms. The resulting ELinux system will permit applications to extend the operating system kernel with the quality management features they require. For instance, a multimedia streaming application may extend the kernel with functions that efficiently monitor available network bandwidth, so that it may then change compression levels and its manner of packet delivery to adjust its operation to resource availability (see Figure 4).

2.4 PDAs and their Use

We are experimenting with a wide variety of portable end user devices. One interesting outcome of our 1999/2000 Yamacraw research is a demonstration of the ability of future home- or local area-based wireless systems to carry interactive, real-time video to palmtop computers, if these devices' screen spaces and communication links are continuously and dynamically managed. Stated more strongly, given the bandwidth, screen space, and computing power of current and near future palm devices, end users can be given real-time video services as long as the underlying service delivery is conscious of the devices' inherent limitations. Such consciousness is not a static quantity. Instead, it involves understanding current user needs (e.g., where is the user currently focusing his or her attention) and exploiting them to operate within resource constraints (e.g., transmit only those portions of a video image that are actually being viewed).

We are now considering additional portable devices, including devices based on wearable computing technologies (wearable Linux machines with head-mounted displays), devices based on low-end technologies like the i-Opener machine (where we will also run the Linux OS), and future palmtop engines. In addition, our project is using mid-range laptops as prototypes of future palmtop or portable devices.

2.5 Advanced Clustering Technologies -- the ASAN Project -- Yalamanchili, Schwan, Schimmel, MacKenzie -- Computing/ECE, Georgia Tech

One project affiliated with the work described here is the ASAN research project conducted jointly by College of Computing and ECE researchers. The aim of this project is to develop the communication technologies for the cluster computers that we expect will drive future community- and home-based computing and data delivery services. The ASAN project is developing hardware/software infrastructures that will enable the provision of quality of service (QoS) guarantees for cluster-level communications (i.e., SANs -- System Area Networks). Two outcomes of our work relevant to the Yamacraw domain will be (1) the creation of cost-effective computing and more importantly, data movement engines that power future home and community services and (2) the extension of the quality management techniques we must employ on the heterogeneous platforms that receive such services to the cluster machines that provide them. In other words, it is our intent to not only deliver services with the quality needed but also to provide them with the quality needed by large numbers of individual clients. In summary, the ASAN project is developing the server-level technologies required for such dynamic service provision.

Example applications or services provided by cluster machines include web servers, video-on-demand servers, immersive environments, virtual meetings, multi-player 3-D games, interactive simulations, and collaborative design environments. Such applications must share the cluster-level network (i.e., its interconnection network), where some of these services are driven by latency requirements, others by requirements like jitter, loss rate, or throughput. The resulting challenge for future cluster servers is to develop a communication infrastructure that effectively manages the clusters' internal network resources to enable the diverse QoS requirements to be met. Consequently, the major elements of the ASAN project are (1) use of powerful, processors embedded in the network interfaces, (2) scheduling paradigms for concurrently satisfying distinct QoS requirements over multiple streams, (3) re-configurable hardware support to enable complex scheduling decisions to be made in the desired time frames, and (4) a flexible and extensible virtual communication machine that provides a uniform interface for dynamically adding hardware/software functionality to the network interfaces. Results of our work are described on the ASAN project web page: http://www.ece.gatech.edu/...
 
 

2.6 External Collaborations -- Power Management, Resource Monitoring, and Cluster-based Service Provision -- University of Georgia -- David K. Lowenthal, Surendar Chandra, Benjamin Bishop

We have begun to collaborate with a group at the Univ. of Georgia that will investigate topics that add the following value to our work. (1) This group will develop representative workloads that are appropriate for performance evaluation studies of mobile, wireless and embedded systems. It seeks to model users' application usage behavior from heterogeneous networks and devices. Specifically, it will focus its efforts on palmtops and mobile laptops. This study will model both system-wide and application-specific behavior. (2) The group will develop methods for program monitoring, specifically using the Palm OS to trace various system calls. This application-specific monitoring will initially evaluate the web-browsing behavior of mobile laptop users by instrumenting a publicly available web browser. (3) The group we will develop system support for interaction between the clients and the server cluster to better utilize both (1) network bandwidth and (2) power on the clients. The server will manage per-client information, as well as periodically communicate this information to all other server machines. Techniques will be investigated to reduce the frequency of this communication without unreasonably sacrificing consistency. Examples of how the client information could be used include sending data in bursts to better utilize client batteries as well as avoiding transmission of parts of data requests when clients have indicated that they generally do not use all of the data. (4) This group will systematically evaluate the techniques described in (2) and (3) using a state of the art, configurable simulator. In particular, it will use the SimpleScalar tool set to measure several important aspects of the client, including details about the memory footprint and cache hit rate. Furthermore, it will modify SimpleScalar to keep track of the switching behavior. Combined, this will allow it to use SimpleScalar to measure the amount of client power consumption.
 
 

3 Student Titles and Abstracts
 
 

Lynn Daley: ``Managing Data Delivery on Heterogeneous Hardware''

In strongly networked environments, software enabling synchronous collaboration among distributed parties has become pervasive. Such support is only slowly making inroads into 'home' environments, where it's high demand is evidenced by the success of facilities such as AOL's Instant Messaging. Limitations in available bandwidths out of homes and to/from wireless systems are one significant problem for synchronous collaboration. A second significant problem is the lack of middleware permitting developers to create services that easily and efficiently deal with dynamic resource variation and limitations across heterogeneous and ubiquitous platforms. This paper presents a model for future 'home' services, outlines a middleware that is the basis for service implementation, and presents a mechanism, controlled channels, that helps developers deal with dynamic resource variation and limitation. Performance evaluation of the controlled channel abstraction demonstrates its feasibility and its utility for selected `home' services.
 
 

Ivan B. Ganev: "Extension Interface for Communication-Related Computations"

This is a vision paper outlining out ideas for an extension infrastructure to allow the placement of application-specific functionality into various levels of the communication data-path.
 
 

Jasmina Jancic: ``Adaptive MPEG Video Streaming in a Wireless Environment''

Wireless environment imposes new challenges on the quality of service for many different applications due to the variability in the performance and reliability of the mobile connection. We concentrate on building an adaptive MPEG player that responds to those changes by dynamically varying the requirements, such as the quality of the MPEG stream or the level of security in the connection. A specific issue addressed by this work is the use of protocol-level performance information (e.g., currently available bandwidth across the wireless media) to improve end user communications. To be addressed by future work is security in the wireless domain.
 
 

Christian Poellabauer: ``ECalls - Flexible Event Delivery for Kernel Extensions''

To gain substantial performance for embedded systems, it must be possible to extend these systems at runtime. That is, rather than requiring that all system functionality is present at all times, it should be possible to configure these systems at runtime, by addition of real-time functionality, by inclusion of application-level code with OS kernels, etc. These facts are well known in the embedded domain, having led to the inclusion of many load-time configuration options for commercial platforms like VxWorks.

Our project seeks to perform fine-grain runtime configuration of embedded software. The idea is to permit even unprivileged users (not operating with superuser privileges) to add or modify services provided by the operating system kernel. Toward these ends, we are developing ELinux, a runtime extensible version of the Linux operating system. The specific mechanism used for extension and for interoperation of kernel extensions with user-level applications is termed ECalls; it permits users to add application-specific event handlers to the kernel, and it implements lightweight synchronous or asynchronous user/kernel communications for use by those handlers. ECalls (Event Calls) addresses several issues in user/kernel communication: events efficiently move from user to kernel and from kernel to user, (1) using multiple methods of control transfer exhibiting alternative real-time behaviors, (2) integrating the scheduling of tasks and of events, (3) using shared memory structures to avoid explicit data passing between kernel services and application, and (4) enabling reductions in the frequency of control transfers with event-specific call filters and delays, and by event batching.
 
 

David Robinson: ``Adaptive, Wireless Event Channels''

In this paper, we describe our efforts to determine the performance behavior of reliable UDP over wireless media. It is well-known that TCP has inadequate performance over wireless media, having resulted in the development of wireless IP and other wireless communication protocols. Our projects aims to place a reliable UDP protocol over a wireless communication layer, the MAC layer of a WaveLAN wireless device used by our group. In addition, we are layering on top of the resulting reliable UDP a middleware layer able to transform and filter ongoing communications. The intent is to build 'smart' wireless data exchange methods, able to adjust to variations in network bandwidth, in available processing power, and in changing user needs. The next steps in our work intend to use Reliable UDP's internal information about communication behavior and performance to enhance the middleware layer's end-to-end performance characteristics. In addition an application will be able to specialize UDP operation in accordance with current application needs. This would make mobile applications adaptable to changing resource needs of applications and permit them to operate in very constrained environments, including in the Personal Digital Assistants used by our project.
 
 

Dong Zhou: ``Enabling Adaptation and Customization in JECho Distributed Event System ''

"JECho is a Java-based communication infrastructure for collaborative high performance applications. JECho's event abstraction supports publish/subscribe programming paradigm and both synchronous and asynchronous event delivery. JECho supports the automatic conversion of Java format events to native format events, and vise versa, and it also provides support for embedded environments through its port to KVM.

This paper introduces JECho<92>s eager handler dynamic adaptation mechanism. Eager handler provides a way of distributing event handling across an event producer/consumer pair. By using handler partitioning, certain classes of applications can potentially reduce the resource requirement on the communication subsystem, off load the amount of event handling computation from resource constrained consumers to producers with more computational resources, and/or devise a consumer specific QOS protocol in an anonymous, multi-consumer environment."
 
 

P. Reid Lyons and Juan Carlos Guzmán: "Latency issues in Wireless LAN"

In this paper we examine the latency issues that plague network communication in its current frontier: palmtop devices. We study this in an asynchronous communication setting, and discuss how it hampers efforts to currently develop applications that depend on process communication. We show sample data on our experiments, and suggest ways to diminish the undesirable effects of latency on communication.
 
 

Juan Carlos Guzmán and P. Reid Lyons:" Streaming Video on a Palmtop"

We present in this paper a protocol used to implement a streaming video application to a palmtop device. Albeit simple, this exercise is prototypical of many applications on these devices, characterized for its marvelous size, its unique limitations of I/O, and the current restrictions on storage, communication and computing power. Successfully programming these devices need to take into account these features, while solving the actual problems. In this application, we were faced with large communication latency that interfered with sending images to the palmtop. Our careful designing of a client/server protocol allows the system to maximize available communication bandwidth, overcoming latency problems associated with the current wireless technology.
 
 

4 Conclusions

The project described in this paper is both delivering some of the basic technologies needed for successful use of distributed wireless and home-wired systems and evaluating these technologies with sample applications. Our goal is to enable rapid creation and also runtime configuration of applications that continuously deliver useful services to end-users. Services are useful when appropriate data is provided at sufficient speeds, when computations are performed in real-time in response to user needs, and when users perceive service delivery to be successful.

The approach we use to continuously deliver useful services is one that exploits changing end user needs as well as system-level knowledge about available resources (both computing and communication resources). That is, when end users change their needs, services are configured to understand such changes, thereby conserving resources that need not be used. Moreover, the underlying system platforms are configured as well to specialize their actions with respect to desired services and to provide service guarantees when and as needed.

This research has produced a variety of artifacts and outcomes, including an extensible version of the Linux kernel, termed ELinux, middleware that enables runtime platform and needs monitoring and consequent service configuration (the ECho and JECho event-based middleware), sample applications utilizing these artifacts, and experimental evaluation on representative target platforms.
 
 

References
 
 

Greg Eisenhauer, Fabian E. Bustamante, and Karsten Schwan, "A Middleware Toolkit for Client-Initiated Service Specialization", Principles of Distributed Computing Middleware Symposium, July 2000.

Greg Eisenhauer. "Portable Self-Describing Binary Data Streams", Technical Report GIT-CC-94-45, College of Computing, Georgia Institute of Technology, Atlanta, GA 30332-0280.

Greg Eisenhauer, Fabian Bustamente and Karsten Schwan, "Event Services for High Performance Computing", To Appear at High Performance Distributed Computing (HPDC-2000).

Greg Eisenhauer and Lynn K. Daley, "Fast Heterogeneous Binary Data Interchange", Proceedings of the 9th Heterogeneous Computing Workshop (HCW 2000), pp 90-101.

Robin Kravets, Ken Calvert, Karsten Schwan. "Payoff Adaptation of Communication for Distributed Interactive Applications". The Journal for High Speed Networking: Special Issue in Multimedia Networking, Winter 1999.

Robin Kravets, Ken Calvert, Karsten Schwan. "Payoff-based communication adaptation based on network service availability". IEEE Multimedia Systems, 1998.

Rajamar Krishnamurthy, Karsten Schwan, and Marcel Rosu, "A Network Co-Processor-Based Approach to Scalable Media Streaming in Servers", International Conference on Parallel Processing, August 2000.

Ling Liu, Calton Pu, Karsten Schwan, Jonathan Walpole, "InfoFilter: Supporting Quality of Service for Fresh Information Delivery", New Generation Computing Journal, Ohmsha, Ltd. and Springer-Verlag, Special issue on Advanced Multimedia Content Processing, Vol.18, No.4, to appear August, 2000.

Beth Plale, Greg Eisenhauer, Lynn K. Daley, Patrick Widener and Karsten Schwan, "Fast Heterogeneous Binary Data Interchange for Event-based Monitoring", To appear at the International Conference on Parallel and Distributed Computing Systems (PDCS2000), August 8-10 2000.

Beth Plale, Greg Eisenhauer, Karsten Schwan, Jeremy Heiner, Vernard Martin, Jeffrey Vetter, "From Interactive Applications to Distributed Laboratories", IEEE Concurrency Volume 6, No. 2, 1998.

Daniela Rosu, Karsten Schwan. "FARACost: An Adaptation Cost Model Aware of Pending Constraints". Proceedings of the 20th IEEE Real-Time Systems Symposium (RTSS), Dec, 1999.

Marcel Rosu and Karsten Schwan, "Support for Recoverable Memory in the Distributed Virtual Communication Machine", International Parallel and Distributed Processing Symposium (IPDPS), May 2000.

D. Silva and K. Schwan, "CTK: Configurable Object Abstractions for Multiprocessors". Technical Report GIT-CC-97-03. Submitted to IEEE Transactions on Software Engineering in January 1997.

Richard West and Christian Poellabauer, "An Optimal, On-Line Window-Constrained Scheduler for Real-Time, Heterogeneous Activities". Technical Report GIT-CC-99-11, College of Computing, Georgia Institute of Technology, Atlanta, GA 30332-0280.

R. West, C. Poellabauer, "Analysis of a Window-Constrained Scheduler for Real-Time and Best-Effort Packet Streams," Real-Time Systems Symposium, 2000.

R. West, C. Poellabauer, K. Schwan, "Scalable Scheduling Support for Loss and Delay Constrained Media Streams, "Proc. 5th Real-Time Technology and Applications Symposium, 1999.

Richard West and Karsten Schwan, "Experimentation with Event-Based Methods of Adaptive Quality of Service Management". Technical Report GIT-CC-99-25, College of Computing, Georgia Institute of Technology, Atlanta, GA 30332-0280.

Richard West and Karsten Schwan, "Dynamic Window-Constrained Scheduling for Multimedia Applications". IEEE International Conference on Multimedia Computing and Systems, 1999.

Richard West, Karsten Schwan, and Christian Poellabauer. "Scalable Scheduling Support for Loss and Delay Constrained Media Streams". Submitted for publication.

Dong Zhou and Karsten Schwan, "Adaptation and Specialization for High Performance Mobile Agents", Proceedings of the 5th USENIX Conference on Object-Oriented Technologies and Systems, 1999.