SCS Faculty Candidate Seminar- Francisco Romero

Name: Francisco Romero, Ph.D, Stanford University

Date: Tuesday, March 12 at 2 p.m.

Location: Marcus Nanotechnology Building, Rooms 1117 & 1118

Title: General Purpose and Interactive Video Analytics

Abstract: The availability of vast video datasets and the increasing accuracy of machine learning models have made exploration of video data an exciting opportunity. Asking complex questions like “Find cases where a car takes a left turn while a pedestrian is crossing the road on a rainy night” over terabytes of videos should be possible. Recent video analytics research expects users will manually reason about their query, combine optimizations, and occasionally train models to meet their performance and accuracy goals. This is a long way from the experience users have when exploring structured data.

In this talk, I will present the design of a general purpose and interactive video analytics system. First, I will present the VIVA video analytics system. VIVA allows users to express domain knowledge about model relationships. VIVA uses this knowledge to automate complex query optimization by deciding how and when it should be applied. Second, I will present Zelda: a system that uses vision-language models to return both relevant and semantically diverse results for top-K video queries. Zelda prompts the VLM with the user’s query in natural language and additional terms to improve accuracy and identify low-quality frames. Zelda improves result diversity by leveraging the rich semantic information encoded in VLM embeddings.

Bio: Francisco Romero currently works at a stealth startup that builds automated intelligence technology for evaluating, searching, and analyzing video for law enforcement, private surveillance, and district attorneys. He recently received his PhD in Electrical Engineering at Stanford University, where his research spanned general machine learning inference, serverless computing, databases, and datacenter scheduling. He has several publications in top-tier systems and database conferences, including a best paper at USENIX ATC 2021.