Welcome!

Art of Defense - ISMAR'08 Demo

Sat, 11 Oct 2008 02:29:57 -0400

Art of Defense explores novel game design and interaction techniques for mobile AR on commodity phone (Nokia N95). The design leverages the phone’s screen size limitations as part of the game challenge. Its dynamic multi-marker building technique enables the game’s mobility and portability, whereas the sketch-based interaction enhances the tangibility and content relationship between the physical and virtual worlds. These techniques encourage both physical movement and tangible interaction with the game pieces -- the two key differentiators between AR and other phone-based games. Art of Defense also demonstrates a new type of AR exploration game, where the players control what parts of the space they can explore.

RSURF Tracking for MAR - ISMAR'08 Demo

Sat, 11 Oct 2008 02:14:26 -0400

This is the result of my 3-month internship at Nokia Research Center, Palo Alto. We present a light-weight tracking method for outdoor augmented reality using pure computer vision technique with a database of geotagged and labeled images.

After extracting and matching SURF features of the first video frame with the database, our method reliably tracks those features locally in the subsequent frames. This method is fast enough to be implemented on mobile devices. Our technique is demonstrated in the context of augmenting a video stream with labels of buildings visible in the video.

Ultimate Commodity

Sun, 15 Jul 2007 19:42:22 -0400

Link to the project website at NTU. This is one of a few traditional theater performances that make use of Augmented Reality technology. The Ultimate Comodity is a play by Daniel Jernigan and based on a dystopic short story by Singapore author, Gopal Barathaman. The story is about a young female scientist’s dream to turn Singapore into an international organ-donor country. So she invented a chemical element from her mother’s DNA that make human’s organ ready to be implanted and decided to put that chemical element secretly into drinking water for all Singapore citizens. However, her mistake in DNA x makes faces of all people who drink that water look very alike. The interesting chaos happen when her father mistook her as her mother and she thinks her father as her boyfriend because all of them look very similar to her mother...

Augmented Reality comes into play to replace the actor and actress’ real head by a model of the mother’s head.

The first demo version was showcased at The Esplanade Theater On the Bay in Singapore on September 2006. In that version, we used marker tracking and overlaid 2D videos of the mother’s face to demonstrate the concept. However, although we tried our best to mitigate the story and play actions so that the performance could be merged well with the technology, the use of markers caused a lot of troubles for the actresses.

We developed the second version in 3 months to show in the Fringe Festival 2007 in Toronto. In that version, we used stereo tracking with 2 Unibrain Fire-i cameras mounted on the ceiling, tracking a 5x5 color-LED boards on top of each actor/actresses’ heads. The rotation is tracked by inertial sensors MTx, and the data is transfered to the computer using Bluetooth.

The graphics is also improved significantly with realistic 3D models of the mother’s head. The static model of the head is constructed by Dr. Quah Chee Kwang from 5 images and a generic 3D head model. After that 3D artists refined the model and generated different models for lip animation for different visemes. My program creates the morphing animation among those viseme models to synchronize with the pre-recorded voice.

The Fringe Festival is famous for exact and limited timing for every show. We have only 2 hours to set up and calibrate the system before the first show, and only 15 minutes before each show to start the system.

One of the important lessons I have learnt is the whole iterative design process -- writing >> technology >> directing >> acting >> and then come back to writing/technology or directing -- which try to integrate the technology difficulties as part of the story itself, or make them invisible as much as possible in the play. We ended up describe in the story that the LED head-mounted markers as important components of the technology, and I was also playing a role in the play as the scientist’s colleague. (I didn’t do anything but sat on the stage to ... control the computer and hand over necessary equipments for the actor/actresses at the correct time!)

Our team includes: Prof. Russell Pensyl (director and producer), Prof. Daniel Jernigan (writer), Gerald Chew (co-director, actor), Debra Teng (actress, the mother), Sara Yang (actress, the young scientist), Anald (stage manager), Kiat (sound and lighting manager), Dr. Quah Chee Kwang (technical advisor), Lee Shang Ping (hardware developer), and me (software developer and tracking technology). Original ideas for the first version was also came from a great couple artists: Diego Diaz and Clara Boj. This projects created many interesting memories with a lot of hard work and collaborations among all team members.

Periphery

Sat, 11 Nov 2006 05:30:25 -0500

Link to the project website. We used 2 infrared cameras to track position of the visitors and their hands, so that when they come closer to the board, the image are changing gradually from the modern scene, which is captured live by the camera behind the board, into the whole forest. when their hands point to the board, another layer of the same scene at some time in the past, drawn by an artist, is revealed at their pointing direction.

The project was shown at The Second International Art and Science Exhibition, Beijing China in November 2006 at Tsinghua University.

Other projects that I was partially involved in

Wed, 11 Oct 2006 06:23:32 -0400

I was involved in Age Invader project, demonstrated in SIGCHI’06.
I was also involved in the vision tracking part for Poultry Internet, which was also demonstrated in SIGCHI’06 and live showed in Good Morning America. That was a very fun trip, staying at a 5-star hotel right at Manhattan New York, and experiencing how the staging of a professional TV broadcasting company look like. That was fantastic!!! (Thank you, Dr. Cheok!)

Free Network Visible Network version 2.0

Mon, 11 Sep 2006 05:41:53 -0400

Link to the main project website. The idea came from a great couple artists: Dr. Diego Diaz and Dr. Clara Boj and I was the developer for the second version. Markers represent for network wireless access points, and we visualize the data packages travelling in the air by transparent boxes and donuts flying between markers in AR. The project was demonstrated in ISEA06 Zero One at San Jose in 2006.

Magic Land

Tue, 2 Aug 2005 05:56:43 -0400

Link to the project website at NUS. This project is the main topic of my Master thesis. Using 3D Live system, we capture and compute the silhouette of the players with 12 Dragonfly cameras put around inside a green room. Then, we render the players’ models in 3D in AR using a modified AR fast version of shape-from-silhouette algorithm to make an interactive table top AR game. Players can see themselves in AR interacting with virtual characters.

The system was demonstrated at various conferences and exhibitions: Planet Games exhibition at Singapore Science Center 9/2004-12/2004, SIGCHI’05, WIRED NextFest at Chicago 06/2005.