Machine Learning Seminar Series Fall 2025 | Pixels to Physics: Understanding and Manipulating Physics from Images

Abstract: A hallmark of human vision is the ability to reason about the physics of the world: we can infer the shape of an object, how light reflects off it, and how the object deforms under force. Yet today’s AI systems still lack this kind of physical intuition. Enabling machines to perceive and manipulate physics would mark a major step toward grounding AI in the real world.

In this talk, I will present my lab’s research at the intersection of computer vision, graphics, and machine learning that takes an inverse graphics perspective: rather than forward-simulating the physics of light, materials, and deformation as in decades of computer graphics, we aim to invert this process to infer and control these properties directly from images. The first part of the talk will focus on explicitly estimating physics, with examples in recovering lighting, reflectance, and object deformation. The second part will show how controlled generative models allow us to manipulate physics, with applications in relighting and simulating facial aging. Together, these efforts highlight a path toward AI systems with a deeper and more actionable understanding of the physical world.

Bio: Roni Sengupta is an Assistant Professor of Computer Science at the University of North Carolina at Chapel Hill and directs the Spatial & Physical Intelligence (SPIN) Lab. Her research lies at the intersection of Computer Vision and Computer Graphics, with a focus on solving Inverse Graphics problems to enable applications in visual content creation and editing, telepresence, AR/VR, robotic perception, and medical imaging. Previously, she was a Postdoctoral Research Associate at the University of Washington, Seattle (2019–2022), following her PhD from the University of Maryland, College Park. She is a recipient of the New & Early Career Trailblazer Award from the National Institute of Biomedical Imaging and Bioengineering (NIH). Her work on Background Matting received a Best Student Paper Honorable Mention at CVPR 2021 (Top 7 out of 1600 accepted papers) and has been adopted by several companies, including Microsoft and Inter-State Studio.

Join Zoom Meeting 
https://gatech.zoom.us/j/94322039610?pwd=ZK7BSj8zuM0U3lylheI9YJalJE3Z3u.1

Meeting ID: 943 2203 9610 
Passcode: 547103