Goal

The point of this project is to explore application-level quality control as often necessary in real-time and embedded systems, particularly when they are mobile. The class of applications considered are data streaming applications, such as streaming video, sensor data, or visualization data. As an example,  consider a wireless video delivery system, where the video is MPEG-encoded on the server and is decoded at the client.  Wireless implies the client may be moving, thus sometimes encountering degradation or loss of data mid-video.  Note that this would not occur if the underlying messaging layer is reliable, but reliable protocols like TCP are known  to have inappropriate behaviors in wireless systems, so you may not have the ability to use TCP. Instead, what you will need to do is provide reliability support at the application level.

Specifically, start with the following. MPEG encoding consists of 3 frame types: I, for major changes in image; B, for less major changes, and P, for smaller changes in image. Frame types all have variable lengths, requiring different decode times. The goal of your work is to avoid or minimze any visible “flutter” or “jitter” in the delivered video, as evident to end users viewing the data. Note also that since we are considering situations in which data is constantly being produced at the source, simple solutions like transferring entire MPEQ files to the client(s) before any of their data is displayed are not feasible.

Step 1)  "Emulating data delivery & recovery over wireless"
Create a client and a server process, on separate processors, communicating thru TCP sockets. Create a third process (thread), the viewer, on the client. Rather than using actual MPEG data, the server should send application-level messages consisting of:
         Message number, Frame Type, and Length;
where Frame Type is randomly generated I, P, or B type, and Length is randomly generated, where length of I > B, and length of B > P.  Deliver each message to the client via the socket, except “drop” random message numbers.
 
The client receives these messages and stores them in a queue, shared with the viewer.  The client checks message numbers, and when one is missing, requests the server resend it.  (Rather than keeping messages numbered higher than the missing one, you should drop messages until the correct “next message” arrives.)
 
Step 2) "Decoding and viewing without flutter"
The viewer removes each message from the queue, and uses the frame type and the length to determine how long to “wait”.  Waiting is to emulate the decode time of an actual MPEG decoder.   Decoding is a high intensity process, and can be slowed by CPU contention as well as data size.  At the end of each wait period,the image frame is ready for display.  For an actual video, the MPEG frame includes the (real) time at which the frame should be displayed.  In this case, assume that a constant display rate is jitter-free, and that a message (frame) is ready for display once it's wait is completed. 
 
Choose a display rate that minimizes the jitter - frames can wait until their display time, but if they are not ready in time, flutter will occur -- ie the image will "freeze" waiting for the next image, then "jump' when the image is ready and displayed.  If the frame is ready early, wait until its display time. 
 
Record the times between your “ready images” and determine when and where the flutter occurs - ie when the image is not ready in time.  Graph the flutter (should be a straight line, with jumps when an image is late).
 
Step 3) If it's a B frame that isn't ready in time, it can be dropped.  Same with a P frame. Dropping a B frame causes a minor flutter dropping a P a more major flutter.  Assign values to these flutters, add frame dropping to your decode algorithm, and run your tests again.  Graph the flutter.
 
Step 4) Discuss the differences between the two graphs.  Would real-time priority scheduling affect the result?  If so, how and why?

Due Date & Turn-In Process

When: Feb 10, 2002, before noon. This is one minute prior to noon on Sunday, Feb 10th. No late assignments will be accepted unless prior arrangements have been made.

Where: To: lkdaley@cc.gatech.edu, Subject: CS 4220 Project II

What: Submit the following in a UNIX "tar" archive.

• The source code (preferably one C/C++ file, one h file, and a Makefile).
• A README file explaining the usage of the program, including any special compiling or testing instructions.
• A report including graphs of your flutter, discussion of the graphs and why they appear the way they do.  (Note Step 4 above!)

Make sure that you do NOT include any binaries in your tar files. We are going to compile your programs from scratch. If you have any special commands needed to compile your code then mention them in your README file.