Augmented Reality

With its scattering of high-end digital video equipment and sophisticated microprocessors, Blair MacIntyre's lab looks like many other university research workspaces around the world.

Except for the zombies.

MacIntyre conducts pioneering research in the field of augmented reality (AR), a new medium that combines aspects of the physical and virtual worlds. An oft-cited but primitive example of AR is the first-down line superimposed on the playing field during televised football games.

"What we're trying to do is directly enhance a person's senses with graphics, sound or other media; the key is that we want to augment what you see, not replace it," says MacIntyre, an associate professor in the School of Interactive Computing. "Another key is that it's an interactive experience. It's not just the first-and-10 line on TV, but it's this combined physical-virtual world that you get to control and explore based on your experience with the physical world."

In recent years MacIntyre's research at the Augmented Environments Lab has focused on the development of handheld AR games, which he believes will build the foundation for other applications. His latest creation, developed in collaboration with the Savannah College of Art and Design, is a shoot-'em-up appropriately called ARHrrrr!!!

The game works like this: A map of an urban intersection is spread over a tabletop and serves as the game venue. When players look at the map through a handheld device (a prototype next-generation smartphone), the device's touch screen shows detailed, three-dimensional buildings flanking the streets, which themselves are populated with strolling civilians and … zombies. By aligning the touch screen's crosshairs with a zombie and pressing on the screen, the player shoots it and scores points. Miss the shot, however, and the zombie starts throwing parts of itself at the players, which register as blood-red splotches on the screen. Zombies may also be eradicated with bombs, which are pieces of Skittles candy the players set on the map and detonate by shooting them. Killing civilians costs points. As players move around the table, their perspectives (as shown on the touch screen display) change accordingly.

"We're experimenting with the 'physicalness' of this system because tangible interaction is important in AR," MacIntyre says. "So in order to aim or move down the street and look around through the AR world, the player has to move."

Augmented Reality

AHRrrrr!!!!, a zombie shoot-em-up game that incorporates AR technology, was designed in Associate Professor Blair MacIntyre’s Augmented Environments Lab in Interactive Computing.

 

Smartphone capabilities make AR possible

An AR system consists of a computer, a camera and other sensors, and a display. For the zombie game, these elements are contained in an NVidia Tegra handheld mobile platform.

Advances in fast mobile technology are making AR relevant and exciting, according to MacIntyre. Smartphones and similar handheld computers already feature GPS, touch sensors, accelerometers and compasses, in addition to the high-end, high-definition video capabilities needed for AR.

"These devices we carry in our pockets have the kind of power we had in laptops just a few years ago," he observes.

Many technical challenges remain before AR takes its place in the mainstream. Tracking precisely where the camera-phone is located relative to the world—a requirement to making the virtual graphics remain in place without moving or "swimming"—needs further work, though MacIntyre points out that the buildings in his zombie game are tightly registered to the map. Real-time modeling of the structure of the physical world, and algorithms for rendering graphics so they integrate more seamlessly with real-world video, need more research as well.

Head-mounted displays will eventually rival the power of handheld devices, MacIntyre predicts, opening up new possibilities for AR including large-scale outdoor applications where a person could be totally immersed in a seamless blend of real and virtual worlds.

AR games and other forms of entertainment have vast potential, but simulations, both medical and military, also could benefit from the technology. New forms of art and expression, innovative teaching techniques, perhaps even a new industry focused on "virtual tourism" may develop.

"AR offers unique opportunities for new kinds of tools, experiences and games when the physical and virtual worlds are tightly integrated," MacIntyre says. "It's exciting to think about the possibilities."