Home   Research   Publications   Teaching   Links   Contact me   Current and Previous Research Projects Video and Fluid Physically Guided Liquid Surface Modeling from Videos We present an image-based reconstruction framework to model real water scenes captured by stereoscopic video. In contrast to many image-based modeling techniques that rely on user interaction to obtain high-quality 3D models, we instead apply automatically calculated physically-based constraints to refine the initial model. The combination of image-based reconstruction with physically-based simulation allows us to model complex and dynamic objects such as fluid. Using a depth map sequence as initial conditions, we use a physically based approach that automatically fills in missing regions, removes outliers, and refines the geometric shape so that the final 3D model is consistent to both the input video data and the laws of physics. Physically-guided modeling also makes interpolation or extrapolation in the space-time domain possible, and even allows the fusion of depth maps that were taken at different times or viewpoints. We demonstrated the effectiveness of our framework with a number of real scenes, all captured using only a single pair of cameras. Physically Based Simulation Water Drops on Surfaces We present a physically-based method to enforce contact angles at the intersection of fluid free surfaces and solid objects, allowing us to simulate a variety of small-scale fluid phenomena including water drops on surfaces. The heart of this technique is a virtual surface method, which modifies the level set distance field representing the fluid surface in order to maintain an appropriate contact angle. The surface tension that is calculated on the contact line between the solid surface and liquid surface can then capture all interfacial tensions, including liquid-solid, liquid-air and solid-air tensions. We use a simple dynamic contact angle model to select contact angles according to the solid material property, water history, and the fluid front's motion. Our algorithm robustly and accurately treats various drop shape deformations, and handles both flat and curved solid surfaces. Our results show that our algorithm is capable of realistically simulating several small-scale liquid phenomena such as beading and flattened drops, stretched and separating drops, suspended drops on curved surfaces, and capillary action. Solving General Shallow Wave Equations on Surfaces We propose a new framework for solving General Shallow Wave Equations (GSWE) in order to efficiently simulate water flows on solid surfaces under shallow wave assumptions. Within this framework, we develop implicit schemes for solving the external forces applied to water, including gravity and surface tension. We also present a two-way coupling method to model interactions between fluid and floating rigid objects. Water flows in this system can be simulated not only on planar surfaces by using regular grids, but also on curved surfaces directly without surface parametrization. The experiments show that our system is fast, stable, physically sound, and straightforward to implement on both CPUs and GPUs. It is capable of simulating a variety of water effects including: shallow waves, water drops, rivulets, capillary events and fluid/floating rigid body coupling. Because the system is fast, we can also achieve real-time water drop control and shape design. Image Processing and Compression Factoring Repeated Content Within and Among Images We reduce transmission bandwidth and memory space for images by factoring their repeated content. A transform map and a condensed epitome are created such that all image blocks can be reconstructed from transformed epitome patches. The transforms may include affine deformation and color scaling to account for perspective and tonal variations across the image. The factored representation allows efficient random-access through a simple indirection, and can therefore be used for real-time texture mapping without expansion in memory. Our scheme is orthogonal to traditional image compression, in the sense that the epitome is amenable to further compression such as DXT. Moreover it allows a new mode of progressivity, whereby generic features appear before unique detail. Factoring is also effective across a collection of images, particularly in the context of image-based rendering. Eliminating redundant content lets us include textures that are several times as large in the same memory space. Light Field Rendering Space-Time Light Field Rendering In this paper, we propose a novel framework called space-time light field rendering, which allows continuous exploration of a dynamic scene in both space and time. Compared to existing light field capture/rendering systems, it offers the capability of using unsynchronized video inputs and the added freedom of controlling the visualization in the temporal domain, such as smooth slow motion and temporal integration. In order to synthesize novel views from any viewpoint at any time instant, we develop a two-stage rendering algorithm. We first interpolate in the temporal domain to generate globally synchronized images using a robust spatial-temporal image registration algorithm followed by edge-preserving image morphing. We then interpolate these software-synchronized images in the spatial domain to synthesize the final view. In addition, we introduce a very accurate and robust algorithm to estimate subframe temporal offsets among input video sequences. Experimental results from unsynchronized videos with or without time stamps show that our approach is capable of maintaining photorealistic quality from a variety of real scenes. Towards Space-Time Light Field Rendering So far extending light field rendering to dynamic scenes has been trivially treated as the rendering of static light fields stacked in time. This type of approaches requires input video sequences in strict synchronization and allows only discrete exploration in the temporal domain determined by the capture rate. In this paper we propose a novel framework, space-time light field rendering, which allows continuous exploration of a dynamic scene in both spatial and temporal domain with unsynchronized input video sequences. In order to synthesize novel views from any viewpoint at any time instant, we develop a two-stage rendering algorithm. We first interpolate in the temporal domain to generate globally synchronized images using a robust spatial-temporal image registration algorithm followed by edge-preserving image morphing. We then interpolate those software-synchronized images in the spatial domain to synthesize the final view. Our experimental results show that our approach is robust and capable of maintaining photo-realistic results.