James James Hays
Associate Professor, School of Interactive Computing, College of Computing, Georgia Institute of Technology
Principal Scientist, Argo AI

My research interests span computer vision, computer graphics, robotics, and machine learning. I work on problems such as object detection, tracking, and localization. My research often involves finding new data sources to exploit (e.g. geotagged imagery) or creating new data sets where none existed (e.g. human sketches, HD maps).

Before joining Georgia Tech, I was the Manning Assistant Professor of Computer Science at Brown University. I was a postdoc at MIT with Antonio Torralba, completed my Ph.D. at Carnegie Mellon University with Alexei Efros, and received my B.S. from Georgia Tech. I am the recipient of the Alfred P. Sloan Fellowship and the NSF CAREER award.

I am on leave for the 2019/2020 Academic year at Argo AI.

Gatech Logo contact
email: hays@gatech.edu
office: CODA 11th floor
Office hours: None while on leave
mail: 756 West Peachtree Street NW, Suite 12E, Atlanta, GA 30308


Students and Collaborators


Ph.D. Students

Graduated Ph.D. Students

Visiting Students

Master's Student Researchers

  • Akash Kumar, Shenhao Jiang, Kapilan Baskar, Vishwas Uppoor alumni: Nitin Kodialbail, Jianan Gao, Govin Vatsan, Vasavi Gajarla, Laura Jeyaseelen, Varun Agrawal, Nate Burnell, Xiaofeng Tao, Chao Qian, Chen Xu, Yipin Zhou, Hang Su, Vibhu Ramani, Paul Sastrasinh, Vazheh Moussavi, Yun Zhang, David Dufresne, Sirion Vittayakorn

Undergraduate Researchers

  • alumni: Wenqi Xian, Cusuh Ham, Lawrence Moore, Sonia Phene, Eric Jang, Hari Narayanan, Sam Birch, Leela Nathan, Eli Bosworth, Jung Uk Kang, Reese Kuppig, Fuyi Huang, Travis Webb


Argoverse: 3D Tracking and Forecasting With Rich Maps.
Ming-Fang Chang*, John Lambert*, Patsorn Sangkloy*, Jagjeet Singh*,
Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, and James Hays.
*co-first authors
CVPR 2019 oral.

Paper, Argoverse project page and data, API code (Github)
ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging.
Samarth Brahmbhatt, Cusuh Ham, Charlie Kemp, and James Hays.
CVPR 2019 oral.

Project page, Blog post
Composing Text and Image for Image Retrieval - An Empirical Odyssey.
Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Jia Li, Fei-Fei Li, and James Hays.
CVPR 2019 oral.

Paper (arXiv), Code (Github)
The Mental Image Revealed by Gaze Tracking.
Xi Wang, Andreas Ley, Sebastian Koch, David Lindlbauer, James Hays, Kenneth Holmqvist, and Marc Alexa.
CHI 2019.

Project Page, ML@GT blog post
Generalization in Metric Learning: Should the Embedding Layer be the Embedding Layer?
Nam Vo and James Hays.
WACV 2019

Paper (arXiv), Code (Github)
Informative Features for Model Comparison.
Wittawat Jitkrittum, Heishiro Kanagawa, Patsorn Sangkloy, James Hays, Bernhard Schölkopf, and Arthur Gretton.
NeurIPS 2018.

Paper (arXiv)
SwapNet: Garment Transfer in Single View Images
Amit Raj, Patsorn Sangkloy, Huiwen Chang, James Hays, Duygu Ceylan, and Jingwan Lu.
ECCV 2018.

Project page, Paper
MapNet: Geometry-Aware Learning of Maps for Camera Localization
Samarth Brahmbhatt, Jinwei Gu, Kihwan Kim, James Hays, and Jan Kautz.
CVPR 2018.

Paper (arXiv)
SketchyGAN: Towards Diverse and Realistic Sketch to Image Synthesis.
Wengling Chen and James Hays.
CVPR 2018.

Paper (arXiv)
TextureGAN: Controlling Deep Image Synthesis with Texture Patches.
Wenqi Xian, Patsorn Sangkloy, Varun Agrawal, Amit Raj, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays.
CVPR 2018.

Paper (arXiv)
On Convergence and Stability of GANs.
Naveen Kodali, Jacob Abernethy, James Hays, and Zsolt Kira.
arXiv, May 2017.

Paper (arXiv)
Revisiting IM2GPS in the Deep Learning Era.
Nam Vo, Nathan Jacobs, and James Hays.
ICCV 2017.

Project Page, Paper (arXiv)
Scribbler: Controlling Deep Image Synthesis with Sketch and Color.
Patsorn Sangkloy, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays.
CVPR 2017.

Project Page, Paper (arXiv), Adobe Max Demo
DeepNav: Learning to Navigate Large Cities.
Samarth Brahmbhatt, James Hays.
CVPR 2017.

Paper (arXiv)
StuffNet: Using 'Stuff' to Improve Object Detection.
Samarth Brahmbhatt, Henrik I. Christensen, James Hays.
WACV 2017.

Paper (arXiv)
Localizing and Orienting Street Views Using Overhead Imagery.
Nam Vo, James Hays.
ECCV 2016.

Project Page, Paper
COCO Attributes: Attributes for People, Animals, and Objects.
Genevieve Patterson, James Hays.
ECCV 2016.


The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies.
Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, James Hays.
Siggraph 2016.

Project Page, Paper

WebGazer: Scalable Webcam Eye Tracking Using User Interactions.
Alexandra Papoutsaki, Patsorn Sangkloy, James Laskey, Nediyana Daskalova, Jeff Huang, James Hays.
IJCAI 2016.

Project Page, Paper

Learning to Match Aerial Images with Deep Attentive Architectures.
Hani Altwaijry, Eduard Trulls, James Hays, Pascal Fua, and Serge Belongie.
CVPR 2016.


Solving Small-piece Jigsaw Puzzles by Growing Consensus.
Kilho Son, Daniel Moreno, James Hays, David B. Cooper.
CVPR 2016.

Project Page, Paper

Tropel: Crowdsourcing Detectors with Minimal Training.
Genevieve Patterson, Grant Van Horn, Serge Belongie, Pietro Perona, James Hays
HCOMP 2015 Best paper runner-up.


Learning Deep Representations for Ground-to-Aerial Geolocalization.
Tsung-Yi Lin, Yin Cui, Serge Belongie, and James Hays.
CVPR 2015 (Oral).


Transient Attributes for High-Level Understanding and Editing of Outdoor Scenes.
Pierre-Yves Laffont, Zhile Ren, Xiaofeng Tao, Chao Qian, and James Hays.
Siggraph 2014.

Project Page, Paper

Good Image Priors for Non-blind Deconvoluton: Generic vs Specific.
Libin Sun, Sunghyun Cho, Jue Wang, and James Hays.
ECCV 2014.

Project Page

Solving Square Jigsaw Puzzles with Loop Constraints.
Kilho Son, James Hays, and David B. Cooper.
ECCV 2014.

Project Page, Paper

Microsoft COCO: Common Objects in Context.
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona,
Deva Ramanan, Piotr Dollar, and C. Lawrence Zitnick.
ECCV 2014.

Project Page, Paper

The SUN Attribute Database: Beyond Categories for Deeper Scene Understanding.
Genevieve Patterson, Chen Xu, Hang Su, and James Hays.
International Journal of Computer Vision. vol. 108:1-2, 2014. Pp 59-81.

Project Page, Paper

Previously published as:
SUN Attribute Database: Discovering, Annotating, and Recognizing Scene Attributes.
Genevieve Patterson and James Hays. CVPR 2012. Paper

Basic level scene understanding: categories, attributes and structures.
Jianxiong Xiao, James Hays, Bryan C. Russell, Genevieve Patterson, Krista A. Ehinger,
Antonio Torralba, and Aude Oliva.
Frontiers in Psychology, 2013, 4:506.

This paper is a survey of recent work related to the SUN database.


Cross-View Image Geolocalization.
Tsung-Yi Lin, Serge Belongie, and James Hays.
CVPR 2013.


FrameBreak: Dramatic Image Extrapolation by Guided Shift-Maps.
Yinda Zhang, Jianxiong Xiao, James Hays, and Ping Tan.
CVPR 2013.

Project Page, Paper

Edge-based Blur Kernel Estimation Using Patch Priors.
Libin "Geoffrey" Sun, Sunghyun Cho, Jue Wang, and James Hays.
ICCP 2013.

Project Page, Paper

Dating Historical Color Images.
Frank Palermo, James Hays, and Alexei A Efros.
ECCV 2012.

Project Page, Paper

How do humans sketch objects?
Mathias Eitz, James Hays, and Marc Alexa.
Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH 2012.

Project Page, Paper

Previously presented as:
Learning to classify human object sketches
Mathias Eitz and James Hays.
ACM SIGGRAPH 2011 Talks Program.

Earlier Research (before 2012)



My research has been funded by a Sloan Fellowship, NSF Career award (1149853), NSF medium 1563727, IARPA's Finder program (FA8650-12-C-7212), and gifts from Google, Microsoft, Pixar, Adobe, and Argo AI.