General Information
Details
My research lies at the intersection of computer vision, robotics, and machine learning, with a primary focus on recognition, scene synthesis, and physical manipulation. I am particularly interested in leveraging novel data sources—from internet-scale geotagged imagery to custom datasets for sketches and grasping—to solve open-world perception problems. My recent work explores autonomous navigation in unstructured off-road environments, 3D scene understanding (including semantic occupancy and scene flow), and audio-visual learning. I aim to develop data-driven systems that not only interpret visual information but can also synthesize content and interact robustly with the physical world.
My teaching philosophy centers on bridging the gap between foundational mathematical intuition and contemporary, practical application. In courses such as Computer Vision, I design curricula that balance classical signal processing techniques with modern deep learning approaches (e.g., Transformers, NeRFs). I prioritize "learning by doing," structuring my courses around substantial, hands-on programming projects that require students to implement algorithms from scratch using industry-standard tools like PyTorch. My goal is to equip students with the rigorous theoretical understanding and the engineering proficiency necessary to tackle current challenges in visual computing and machine learning.
GaussianFormer3D: Multi-Modal Gaussian-based Semantic Occupancy Prediction with 3D Deformable Attention.
Lingjun Zhao, Sizhe Wei, James Hays, and Lu Gan.
ICRA 2026.
Clink! Chop! Thud! — Learning Object Sounds from Real-World Interaction.
Mengyu Yang, Yiming Chen, Haozheng Pei, Siddhant Agarwal, Arun Balajee Vasudevan, and James Hays.
ICCV 2025.
Uncertainty-aware Accurate Elevation Modeling for Off-road Navigation via Neural Processes.
Sanghun Jung, Daehoon Gwak, Byron Boots, and James Hays.
CoRL 2025.
OmniNOCS: A unified NOCS dataset and model for 3D lifting of 2D objects.
Akshay Krishnan, Abhijit Kundu, Kevis-Kokitsi Maninis, James Hays, and Matthew Brown.
ECCV 2024 Oral.
I Can't Believe It's Not Scene Flow!
Ishan Khatri*, Kyle Vedder*, Neehar Peri, Deva Ramanan, and James Hays.
(* equal contributions)
ECCV 2024.
What Matters in Range View 3D Object Detection.
Benjamin Wilson, Nicholas Autio Mitchell, Jhony Kaesemodel Pontes, and James Hays.
CoRL 2024.
Shelf-Supervised Multi-Modal Pre-Training for 3D Object Detection.
Mehar Khurana, Neehar Peri, Deva Ramanan, James Hays.
CoRL 2024.
Granular Privacy Control for Geolocation with Vision Language Models.
Ethan Mendes, Yang Chen, James Hays, Sauvik Das, Wei Xu, Alan Ritter.
EMNLP 2024.