This paper introduces interaction mechanisms for conveying temporal characteristics of time-varying volume data based on temporal styles. We demonstrate the flexibility of the new concept through different temporal style transfer function types and we define a set of temporal compositors as operators on them. The data is rendered by a multi-volume GPU raycaster that does not require any grid alignment over the individual time-steps of our data nor a rectilinear grid structure. The paper presents the applicability of the new concept on different data sets from partial to full voxel alignment with rectilinear and curvilinear grid layout.

In this paper we present a project which aimed at virtually reconstructing the impressive (7x11 m.) portal of the Ripoll Monastery, Spain. The monument was acquired using triangulation laser scanning technology, producing a dataset of more than 2000 range maps for a total of more than 1 billion triangles. All the steps of the entire project are described, from the acquisition planning to the final setup for the dissemination to the public. In particular, we show how time-of-flight laser scanning data can be used to obtain a speed up in the alignment process, and how, after model creation and imperfections repairing, an interactive and immersive setup gives the public the possibility to navigate and visualize the high detail representation of the portal.

Advances in range measurement devices in recent years have opened up new opportunities and challenges for fast 3D modeling of large scale environments. Applications of such technologies include virtual walk and fly through, urban planning, disaster management, object recognition, training, and simulations. In this paper, we present a general framework for surface reconstruction and segmentation using partially ordered 3D point clouds composed of registered ground-based and airborne range and color data. Our algorithms can be applied to a large class of LIDAR data acquisition systems, where ground-based data is obtained as a series of scan lines. We develop an efficient and scalable algorithm that reconstructs surfaces and segments ground-based range data simultaneously. We also propose a new algorithm for merging ground-based and airborne meshes which exploits the locality of the ground-based mesh. We demonstrate the effectiveness of our results on data sets obtained by two acquisition systems.

Packet loss affects the receiving quality of 3D meshes transmitted over unreliable networks. While some applications are able to tolerate higher loss, others may need to restrict the loss below a specified level. In this work we describe a curvature-driven probabilistic strategy to control the adverse impact of packet loss. Critical mesh features, with high curvature, like sharp edges and corners are allocated more bandwidth to increase the rate of their successful transmission. When the probability of visual degradation exceeds an acceptable level, a group of curvature indices is added to the transmission pipeline. The size of the indices is governed by three parameters: The mesh resolution, the minimum required quality and the tolerance. We incorporate this new strategy with an earlier interleaved transmission approach. Experimental results show that the reconstructed meshes using the integrated strategy have higher visual quality.

We present an event-based scheduling method for time-critical collision detection that meets real-time constraints by balancing and prioritizing computation spent on intersection tests without starvation. Our approach tests each potentially colliding pair of objects at a different frequency, with unbounded temporal resolution. We show that believability is preserved by adaptively prioritizing intersection tests to reduce errors in collision detection, using information about the objects and scene. Through the combination of kinetic sweep and prune with stride scheduling we continuously interleave rendering, broad phase collision pruning, narrow phase intersection testing, and collision response. This approach accrues no per-frame overhead and allows interruption at any point in collision detection, including the broad phase.

Surface light fields can be used to render the complex reflectance properties of a physical object. One limitation is that they can only represent the fixed lighting conditions of the environment where the model was captured. If a specific lighting condition is desired, then there are two options: either use a combination of physical lights as an approximation, or capture a full 6D surface reflectance field and only use the portion that corresponds to the desired lighting. In this paper we present a method for capturing a surface light field using the virtual illumination from an environment map. We use a simple setup consisting of a projector, a camera, a pan-tilt unit, and tracking fiducials to recreate the desired lighting environment. To decrease noise and improve the quality of thecapture under low- and high-dynamic range environment maps, we use an extended version of the multiplexed illumination algorithm. We show results from objects captured under different lighting environments.

Using ear shape as a biometric trait is one of the most recent trends in the biometric research communities. In this work, a fully automatic and fast technique based on the AdaBoost algorithm is used to detect the human ear from 2D and the corresponding 3D profile image. A modified Iterative Closest Point (ICP) algorithm is then used for matching the extracted ear shapes. The ICP algorithm is applied hierarchically first on a lower and then on a higher resolution meshes of 3D ear data. We obtain a rank one ear recognition rate of 93% on the profile images of the University of Notre Dame biometrics database. The proposed recognition approach does not require any manual intervention or sharp extraction of ear contour from the detected ear region. Moreover, the system performance does not rely on the presence of a particular feature of the ear.

In this work we present a method to detect overlaps in image sequences, and use this information to integrate overlapping sparse 3D structure from video sequences. The additional temporal information of these images is used to increase robustness over single image pair matching. A scanline optimization problem formulation is used to compute the best sequence alignment using wide-baseline image matching techniques. Compared to a direct dynamic programming approach, the scanline optimization formulation increases the robustness of sequence alignment for general relative motions. The proposed alignment method is employed to integrate sparse 3D models reconstructed from separate video sequences. In addition loop closures are detected. Consequently, the 3D modeling process from sequential image data can be split into fast sequence processing and subsequent global integration steps.

In this paper, we develop a novel method for camera calibration and correction. The novel method first employs a rigidity constraint from the rigid rotation matrix and the radial alignment constraint from the pin-hole camera model to estimate both camera intrinsic and extrinsic parameters with a closed-form solution without considering the camera distortion. Then the well-known Levernburg-Marquardt (LM) algorithm is employed to optimize the parameters of interest in two steps: the first step optimizes the first order radial distortion coefficient and the z component of the camera position using a partial image formation model, and the second step optimizes all the parameters of interest using a complete image formation model: 4 intrinsic, 7 extrinsic and 4 distortion parameters. The LM algorithm is initialised either as the parameters estimated so far or as zero. The optimization is achieved through minimising the sum of the squared differences between the distorted projected 3D world control points and their given corresponding distorted image points. The distorted points are finally corrected using again the LM algorithm initialized by the distorted image points themselves, minimizing the squared difference between the distorted corrected point and the given distorted image point. The experimental results based on both synthetic data and real images show that the proposed algorithm produces promising camera calibration and correction results.

Stereo vision is one of the most intensive and challenging problems in computer vision. It makes use of stereo cues to extract 3D information from 2D images. Besides stereo cues, there are some statistical learning based approaches which exploit monocular cues to predict the underlying 3D structure. As for human vision, the amazing ability for 3D interpretation is based on the combination of these two kinds of cues. Therefore, in this paper, we make an attempt to incorporate monocular cues into the stereo matching system. A two-level graph is utilized to fuse the low-resolution monocular cues and high-resolution stereo cues together. Then the optimal labeling results are calculated via graph-cuts. The experiment results show that we can obtain more accurate disparity map than is possible using either monocular or stereo cues alone.

In this paper, we propose to rely only on images to generate novel views, and recall why modeling of the complete scene is often too expensive in the context of view interpolation. We investigate ways to achieve view interpolation by mean of forward and backward mapping. We present situations in which each one requires less computations and gives better results. Contrary to what we might expect, very simple stereo algorithms can produce very convincing interpolation despite providing really bad disparity maps. We propose to explain this with a probabilistic model of depth discontinuities. We test this model on synthetic data created to fit real image statistics and compare with images widely used in stereo. In practice, forward and backward mapping methods can rely on simple stereo algorithms running in real time, to produce very good results. A sequence of real images was acquired to allow accurate comparison of interpolated images, and standard metrics are used to assess the quality.

This work describes a method for filling holes in a 3D mesh based on 2D image restoration algorithms. Since these algorithms need an image as input, the first stage of the method concerns a 3D to 2D transformation for a range image creation. The image restoration algorithms are applied to it. Once the image has been repaired, the inverse transformation 2D to 3D is performed and the repaired 3D surface recovered. To test the method, artificial holes have been generated on a set of 3D surfaces. The goodness of the results has been measured from the comparison between the 3D original surfaces and the 3D repaired ones. An evaluation with commercial software has been carried out to show the validity of the method. The image restoration algorithms have been applied to 3D cultural heritage modeling with good results.

Many commercially available 3D sensors suitable for face image capture employ either passive (or texture-assisted) stereo imaging or structured illumination with a moving stripe. Both of these techniques require a stationary subject. We describe an initial design and evaluation of a fixed-stripe, moving object 3D scanner designed for human faces. Our method of acquisition requires the subject to walk through a light screen generated by two laser line projectors. Triangulation and tracking yield a 3D image of the subjects face from multiple images. To demonstrate the accuracy of our initial design, a small-scale facial recognition experiment was executed. In an experiment involving 13 subjects with 4 images per subject, we achieve 92.3% rank one recognition using an Iterative Closest Point (ICP) based matching method, demonstrating the feasibility of the technique.

We present a new geometric calibration method for a structured light system combining a projector with a camera, using a planar target with circular control points. By solely exploiting the mapping between projected conics, the proposed method is strictly geometric and provides unbiased camera to projector correspondences during its application. Such a geometric method does not rely on radiometric calibration. Moreover, the method consistently ensures uniform coverage of the working volume and automatically avoids interference between both the projected and the printed patterns on the calibration target.

Acquiring the 3D mesh of a scene from stereo images is a major task in computer vision. It usually involves several steps including stereo matching and meshing. Unfortunately, the time required to generate the 3D mesh is time demanding due to the large amount of pixels to be processed. In this work, we propose a framework to accelerate the overall process. The key issue is to first reduce the number of pixels by approximating an image with a content adaptive mesh. The nodes of the mesh are sparse and they represent the non-uniform samples of the image. To benefit from the reduced set of pixels, we formulate a dynamic programming based stereo matching algorithm which computes the depth only at the sparse samples.

We introduce a method which uses the light emitted by a computer screen to illuminate an object such as a human face from multiple directions, simultaneously capturing images with a webcam in order to perform photometric stereo. Dominant eigenvectors of the captured images provide surface normals, which are integrated into a 3D surface using Gauss-Seidel relaxation. The system runs at 10 frames per second on a consumer laptop computer.

In recent years, stereoscopic technology has advanced from stereoscopic to autostereoscopic displays. These latter family involves to display several views of a scene. In the case of real-time computer graphics images, the standard approach consists in rendering every view independently. This paper presents an alternative method to generate multiple views for autostereoscopic displays in a single rendering pass. Our algorithm is based on the fact that vertices properties remain the same from different viewpoints. Taking advantage of the latest generation of GPUs including geometry shaders, we propose a method that significantly speeds up the rendering process by duplicating and transforming incoming primitives for a defined set of views. Our method involves very few modifications to be used with a standard stereo device.

Whether interacting with a Collaborative Virtual Environment, or CVE, locally or one networked across the Internet, any delay in the system can lead to a reduced sense of immersion. Input sensor delay and network delay are two common problems in CVE design that can be overcome with the application of prediction algorithms to the system. The purpose of this experiment was to assess the quality of feed forward back propagation neural networks in predicting natural avatar arm movement used in a CVE. In addition the experiment attempted to find the bounds for precise neural network prediction. The results show many different combinations of back propagation neural network topologies are capable of predicting up to 400 ms of human arm movements relatively accurately.

We propose a new method for 3D-mesh model characteristic view selection. It consists in using the views that come from the equilibrium states of a 3D-model: they correspond to the horizontal plane on which an object is statically laying under the effect of gravity. The selected views are then very intuitive for the user. Indeed, to present a query, the user will take a photo or draw a sketch of the object on a table or on a floor, putting thus the object in a static mechanical equilibrium. Consequently, our view selection method follows the same principles: finding all the equilibrium planes of an object and obtaining their relative 2D views. We present the experiments and results of our method on the Princeton 3D Shape Benchmark Database using a collection of 50 images (photos, sketches, etc.) as queries, showing the performance of our method in 3D retrieval from photos.

In this paper, we present a new technique for solving the difficult problem of estimating the axis of symmetry for axially-symmetric surfaces. Accurate solutions to this problem are important in archaeology for systems that seek to reconstruct pottery vessels from measurements of their fragments. Our approach estimates quadratic surfaces at each measured surface point and uses a Markov Random Field superimposed on the measured surface mesh to estimate a collection of surface patches, each of which lies close to a single 3D quadratic surface. For each surface patch we estimate an quadratic implicit polynomial whose coefficients directly provide an estimate of the unknown axis location and orientation. Competing estimates of the global axis are combined using a Maximum Likelihood Estimation (MLE) framework that reflects the uncertainty present in the estimates computed from each surface patch. Our approach differs from past approaches by combining estimates derived from large surface regions that include many measurements instead of combining many local (often pointwise) estimates of the surface to determine the global estimate. Estimates from these large regions are more robust to noise and have sufficient data to generate statistics that accurately reflect the uncertainty in the computed estimates. As such, each estimate of the central axis is less susceptible to outliers and the overall axis estimate is significantly improved.

This paper deals with the automatic stereo reconstruction of wide-area scenes. Its particular goal is a computationally efficient method that can be performed on a personal computer, despite the large amount of data involved in the reconstruction of wide-area scenes. Robustness is considered in terms of the accuracy of the final reconstruction, as well as, in the context of simplifying the image acquisition process for the end-user.

Reconstructing the 3D surface from a set of provided range images -- acquired by active or passive sensors -- is an important step to generate faithful virtual models of real objects or environments. Since several approaches for high quality fusion of range images are already known, the runtime efficiency of the respective methods are of increased interest. In this paper we propose a highly efficient method for range image fusion resulting in very accurate 3D models. We employ a variational formulation for the surface reconstruction task. The global optimal solution can be found by gradient descent due to the convexity of the underlying energy functional. Further, the gradient descent procedure can be parallelized, and consequently accelerated by graphics processing units. The quality and runtime performance of the proposed method is demonstrated on well-known multi-view stereo benchmark datasets.

We consider efficient 3D shape acquisition and surface registration using dissimilar laser range scanners. In this paper, we exploit the fundamental 3D scanning trade-off between the coverage of the global shape structure and numerous surface patches to construct a hybrid laser scanning system provided that it can acquire both global and local shape information. The scanned low-resolution global shape data supplies the global shape structural prior for registering the high-resolution local 3D surface patches. Local surface patches can thus be optimally registered requiring less overlapping and thus reducing redundancy. To verify the feasibility of our hybrid 3D laser scanning system, we have implemented a prototype based on two laser range scanners, a hand-held one for the coarse global low-resolution model and a second stationary high-resolution line scanning system. This prototype system was evaluated for various real 3D models. Based on geometric data alone without using texture information, the results show that the proposed hybrid 3D scanning system outperforms previous approaches to the 3D acquisition and surface registration problem. The approach can be further extended and applied to other practical 3D shape applications.