James James Hays
Associate Professor, School of Interactive Computing, College of Computing, Georgia Institute of Technology

My research interests span computer vision, robotics, and machine learning. I work on problems related to recognition, synthesis, and manipulation. My research often involves finding new data sources to exploit (e.g. geotagged imagery) or creating new data sets where none existed (e.g. sketches or grasps).

Before joining Georgia Tech, I was the Manning Assistant Professor of Computer Science at Brown University. I was a postdoc at MIT with Antonio Torralba, completed my Ph.D. at Carnegie Mellon University with Alexei Efros, and received my B.S. from Georgia Tech. I am the recipient of the Alfred P. Sloan Fellowship, the NSF CAREER award, and the PAMI Mark Everingham Prize. From 2017 to 2022 I was a Principal Scientist at Argo AI researching self-driving vehicle perception. I am currently working with Overland AI on off-road autonomous vehicles.

Gatech Logo contact
email: hays@gatech.edu
office: CODA S1155B
mail: 756 West Peachtree st NW, Suite 12E
Atlanta, GA 30308

Teaching

Students and Collaborators

Ph.D. Students

Former Ph.D. Students

Previous Postdoc

  • Pierre-Yves Laffont now at Meta

Visiting Students

Master's Student Researchers

  • Akash Kumar, Shenhao Jiang, Kapilan Baskar, Vishwas Uppoor alumni: Nitin Kodialbail, Jianan Gao, Govin Vatsan, Vasavi Gajarla, Laura Jeyaseelen, Varun Agrawal, Nate Burnell, Xiaofeng Tao, Chao Qian, Chen Xu, Yipin Zhou, Hang Su, Vibhu Ramani, Paul Sastrasinh, Vazheh Moussavi, Yun Zhang, David Dufresne, Sirion Vittayakorn

Undergraduate Researchers

  • alumni: Wenqi Xian, Cusuh Ham, Lawrence Moore, Sonia Phene, Eric Jang, Hari Narayanan, Sam Birch, Leela Nathan, Eli Bosworth, Jung Uk Kang, Reese Kuppig, Fuyi Huang, Travis Webb

Recorded Talks

Highlighted Recent Papers

The Un-Kidnappable Robot: Acoustic Localization of Sneaking People.
Mengyu Yang, Patrick Grady, Samarth Brahmbhatt, Arun Balajee Vasudevan, Charles C. Kemp, and James Hays.
ICRA 2024.

Project page, arXiv
ZeroFlow: Fast Zero Label Scene Flow via Distillation.
Kyle Vedder, Neehar Peri, Nathaniel Chodosh, Ishan Khatri, Eric Eaton, Dinesh Jayaraman, Yang Liu, Deva Ramanan, and James Hays.
ICLR 2024.

Project page, arXiv
PressureVision++: Estimating Fingertip Pressure from Diverse RGB Images.
Patrick Grady, Jeremy A. Collins, Chengcheng Tang, Christopher D. Twigg, Kunal Aneja, James Hays, and Charles C. Kemp.
WACV 2024.

arXiv, WACV paper
Lidar Panoptic Segmentation and Tracking without Bells and Whistles.
Abhinav Agarwalla, Xuhua Huang, Jason Ziglar, Francesco Ferroni, Laura Leal-Taixe, James Hays, Aljosa Osep, and Deva Ramanan.
IROS 2023.

Project page, arXiv
Learning Lightweight Object Detectors via Multi-Teacher Progressive Distillation.
Shengcao Cao, Mengtian Li, James Hays, Deva Ramanan, Yu-Xiong Wang, and Liangyan Gui.
ICML 2023.

ICML paper
Soft Augmentation for Image Classification.
Yang Liu, Shen Yan, Laura Leal-Taixe, James Hays, Deva Ramanan.
CVPR 2023.

CVF Page
Modulating Pretrained Diffusion Models for Multimodal Image Synthesis.
Cusuh Ham, James Hays, Jingwan Lu, Krishna Kumar Singh, Zhifei Zhang, Tobias Hinz.
Siggraph 2023 Conference.

Project Page
LANe: Lighting-Aware Neural Fields for Compositional Scene Synthesis.
Akshay Krishnan, Amit Raj, Xianling Zhang, Alexandra Carlson, Nathan Tseng, Sandhya Sridhar, Nikita Jaipuria, James Hays.
arXiv, April 2023.

arXiv
Visual Estimation of Fingertip Pressure on Diverse Surfaces using Easily Captured Data.
Patrick Grady, Jeremy A. Collins, Chengcheng Tang, Christopher D. Twigg, Kunal Aneja, James Hays, Charles C. Kemp.
arXiv, January 2023.

arXiv
Far3Det: Towards Far-Field 3D Detection.
Shubham Gupta, Jeet Kanjani, Mengtian Li, Francesco Ferroni, James Hays, Deva Ramanan, Shu Kong.
WACV 2023.

CVF page
PressureVision: Estimating Hand Pressure from a Single RGB Image.
Patrick Grady, Chengcheng Tang, Samarth Brahmbhatt, Christopher D. Twigg, Chengde Wan, James Hays, and Charles C. Kemp.
ECCV 2022 Oral.

arXiv
A Sketch is Worth a Thousand Words: Image Retrieval with Text and Sketch.
Patsorn Sangkloy, Wittawat Jitkrittum, Diyi Yang, and James Hays.
ECCV 2022.

arXiv
CoGS: Controllable Generation and Search from Sketch and Style.
Cusuh Ham, Gemma Canet Tarres, Tu Bui, James Hays, Zhe Lin, and John Collomosse.
ECCV 2022.

Project page, arXiv
SALVe: Semantic Alignment Verification for Floorplan Reconstruction from Sparse Panoramas.
John Lambert, Yuguang Li, Ivaylo Boyadzhiev, Lambert Wixson, Manjunath Narayana, Will Hutchcroft, James Hays, Frank Dellaert, and Sing Bing Kang.
ECCV 2022.
Argoverse 2: Next Generation Datasets for Self-driving Perception and Forecasting.
Benjamin Wilson, William Qi, Tanmay Agarwal, John Lambert, Jagjeet Singh, Siddhesh Khandelwal, Bowen Pan, Ratnesh Kumar, Andrew Hartnett, Jhony Kaesemodel Pontes, Deva Ramanan, Peter Carr, and James Hays
NeurIPS Datasets and Benchmarks 2021

Argoverse 2 home page, NeurIPS page
Trust, but Verify: Cross-Modality Fusion for HD Map Change Detection.
John Lambert and James Hays.
NeurIPS Datasets and Benchmarks 2021

Argoverse 2 home page, NeurIPS page
ANR: Articulated Neural Rendering for Virtual Avatars.
Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, and Christoph Lassner.
CVPR 2021.

Project page, arXiv
PVA: Pixel-aligned Volumetric Avatars.
Amit Raj, Michael Zollhoefer, Tomas Simon, Jason Saragih, Shunsuke Saito, James Hays, and Stephen Lombardi.
CVPR 2021.

Project page, arXiv
Scene Flow from Point Clouds with or without Learning.
Jhony Kaesemodel Pontes, James Hays, Simon Lucey.
3DV 2020 Oral.

Project page, arXiv
3D for Free: Crossmodal Transfer Learning using HD Maps.
Benjamin Wilson, Zsolt Kira, and James Hays.
arXiv preprint arXiv:2008.10592, August 2020.

arXiv
Tide: A general toolbox for identifying object detection errors
Daniel Bolya, Sean Foley, James Hays, Judy Hoffman.
ECCV 2020 spotlight.

Project page, arXiv
ANR: Articulated Neural Rendering for Virtual Avatars.
Amit Raj, Julian Tanke, James Hays, Minh Vo, Carsten Stoll, Christoph Lassner.
arXiv preprint arXiv:2012.12890, December 2020.

Project page, arXiv
Computational discrimination between natural images based on gaze during mental imagery.
Xi Wang, Andreas Ley, Sebastian Koch, James Hays, Kenneth Holmqvist, and Marc Alexa.
Scientific Reports, August 2020.
Open Access Article

Related earlier conference paper:

The Mental Image Revealed by Gaze Tracking.
Xi Wang, Andreas Ley, Sebastian Koch, David Lindlbauer, James Hays, Kenneth Holmqvist, and Marc Alexa.
CHI 2019.

Project Page, ML@GT blog post
ContactPose: A Dataset of Grasps with Object Contact and Hand Pose.
Samarth Brahmbhatt, Chengcheng Tang, Chris Twigg, Charlie Kemp, and James Hays
ECCV 2020.

Project page, arXiv
MSeg: A Composite Dataset for Multi-domain Semantic Segmentation.
John Lambert, Zhuang Liu, Ozan Sener, James Hays, and Vladlen Koltun.
CVPR 2020.

Project page, Paper
ContactGrasp: Functional Multi-finger Grasp Synthesis from Contact
Samarth Brahmbhatt, Ankur Handa, James Hays, and Dieter Fox
IROS 2019.

Project page, arXiv paper
Towards Markerless Grasp Capture.
Samarth Brahmbhatt, Charlie Kemp, and James Hays.
CVPR 2019 CV for AR/VR Workshop.

Project page, arXiv
Argoverse: 3D Tracking and Forecasting With Rich Maps.
Ming-Fang Chang*, John Lambert*, Patsorn Sangkloy*, Jagjeet Singh*,
Slawomir Bak, Andrew Hartnett, De Wang, Peter Carr, Simon Lucey, Deva Ramanan, and James Hays.
*co-first authors
CVPR 2019 oral.

Paper, Argoverse project page and data, API code (Github)
ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging.
Samarth Brahmbhatt, Cusuh Ham, Charlie Kemp, and James Hays.
CVPR 2019 oral and best paper finalist.

Project page, Blog post
Composing Text and Image for Image Retrieval - An Empirical Odyssey.
Nam Vo, Lu Jiang, Chen Sun, Kevin Murphy, Jia Li, Fei-Fei Li, and James Hays.
CVPR 2019 oral.

Paper (arXiv), Code (Github)
Generalization in Metric Learning: Should the Embedding Layer be the Embedding Layer?
Nam Vo and James Hays.
WACV 2019

Paper (arXiv), Code (Github)
Revisiting IM2GPS in the Deep Learning Era.
Nam Vo, Nathan Jacobs, and James Hays.
ICCV 2017.

Project Page, Paper (arXiv)
Scribbler: Controlling Deep Image Synthesis with Sketch and Color.
Patsorn Sangkloy, Jingwan Lu, Chen Fang, Fisher Yu, and James Hays.
CVPR 2017.

Project Page, Paper (arXiv), Adobe Max Demo

The Sketchy Database: Learning to Retrieve Badly Drawn Bunnies.
Patsorn Sangkloy, Nathan Burnell, Cusuh Ham, James Hays.
Siggraph 2016.

Project Page, Paper

Earlier Papers (click to expand)

Support

My research has been funded by a Sloan Fellowship, NSF Career award (1149853), Sandia National Labs, NSF medium 1563727, IARPA's Finder program (FA8650-12-C-7212), and gifts from Intel, Google, Microsoft, Pixar, Adobe, and Argo AI.