Data and Software:

     The images used in this project were extracted from a set of ultrasound diagnostic images created by a Hewlett Packard proprietary system. The first goal was to isolate an image plane which showed the relevant structures and extract that plane out of the volume set for each time instance. This was accomplished by using a C program based on the data structure of the HP system, viewing data volumes and slices in IRIS Explorer, and rewriting the C program to extract only the relevant data slices. (This was accomplished with the help of a cardiologist, as it is quite difficult for a mere layman to identify appropriate structures.)
    Since a significant amount of noise was present in the images, a filtering program was written in MATLAB to remove the extraneous noise while preserving as much of the actual data as possible. Median filtering combined with a simple adaptive filtering method were used towards this end, and the program was optimized so that one set of parameters could adequately filter all of the images.
    The total number of images in one sequence is fifteen, representing one complete heartbeat. Ideally, several sequences will be to train the final program, and the program will be tested on a few new sequences. Considering the small number of images and the techniques being used, I have opted to do a majority of this work using MATLAB. The only caveat to this is the possible use of optical flow, which may be done with compiled code if necessary.

Goals:

    Short Term:
 
        1) Extract significant edges which define the borders of the ventricle

        2) Apply motion analysis associate edge points with specific structures (wall, valve, septum)

        3) Associate parameters with the motion of the specific structures

    Long Term:

        1) Redefine the edges based on motion parameters (using active contours)
 
        2) Automatically determine parameters based on training sequences and test on new images

        3) Extract borders for all slices of a 3-D data set, and render the various structures (TIME PERMITTING)

Progress:

    At this point, I have been able to automatically extract the borders of the ventricles of each image using an edge detection algorithm and various filtering techniques. The largest edge in each image is isolated from all extraneous (for the time being, anyway) edges, and the points are written to an array. A normal vector is then calculated for each point in the array. The ventricular borders can be seen in the following image sequence:

These should be animated once I get a chance to use the ECE labs.

    Motion analysis can be performed in a number of ways. The first attempt has been with optical flow, but has failed to produce usable results.
This appears to be due to the non rigid motion of the heart structures, combined with the fact that various features move in and out of the image plane throughout the sequence. It can be seen that the intraventricular septum moves out of the plane considerably, due to a defect of this structure. If it were to remain intact, it would separate the left and right side of the ventricles (the black portion of the images), and we could consider each ventricle separately. This would be easier from a computer vision point of view, but not very relevant in a clinical environment, as it is not very interesting to examine healthy hearts.
    Currently, I am using image differencing to characterize motion. A background image is obtained by averaging all the images. and each image is subtracted from the background to highlight the areas of motion. The motion at each point along the border is quantified by adding the motion value (the difference between that pixel and the background) of several pixel in the neighborhood of the edge point. (The pixels used are determined based on the normal vector at that point.) This method has proven to be useful in isolating the septum, which paves the way for future work.
    The septum ( and in many cases the lack thereof) has been a hindrance for me because it makes it difficult to correlate edge points in neighboring sequences. Having isolated it, the next step is to perform this correlation so that specific points can be tracked between images, and the motion can be parameterized in order to segment the structures. Also, frequency domain analysis of the motion value may prove useful in further segmenting the data, because the wall motion is more organized  and predictable. this analysis is still under investigation.
     Finally, I am also experimenting with a region-based approach to motion characterization, using user-defined regions which may be labeled as belonging to various structures. The purpose of this its to isolate unique characteristics of the data regions, which will aid in the development of motion parameters. This work is also still under investigation.

Notice the highlighted area; this is the septum.

Future Objectives:

    I expect that the various motion parameters being tested will yield appreciable results very soon, which will enable me to move on to the long term objectives. Since the initial contours defined are in fact a crude estimation, they will need ot be refined using active contours. Once segmentation is performed, the main border can then be broken up into its components, and a separate process for each piece can be used to define the borders of the structures of interest. The method for determining the borders will be based on the motion characteristics of that particular structure.