Researchers Use GeoGuessr Champion to Test Geolocation Accuracy in VLMs

The house in the distance, with a red, hip-shaped roof and white walls, tells Radu Casapu that this place is probably somewhere in Spain or Portugal. 

The surrounding trees resemble those of a eucalyptus forest, which could indicate northern Portugal or the Spanish region of Galicia.

It’s the signposts on the road that give it away. They are flat and wide, which is common in Spain but not in Portugal.

Casapu, a master’s student in Georgia Tech’s School of City and Regional Planning, correctly reasons that the picture of a road he’s looking at is in Galicia.

Give Casapu a photo, and he will likely be able to tell you where it was taken. 

“I start with infrastructure clues that are specific to a country, region, state or province,” Casapu said. “They include roads or electricity poles, which often remain consistent throughout a country. Once you narrow down the country, you can use more specific factors like vegetation, specific landscapes, or architecture, because these are very nuanced. It’s a top-down approach.”

This is why Casapu is the reigning GeoGussr World Champion — and the ideal expert to test vision-language models (VLMs) on how good they are at geolocation.

GeoGuessr is a geography browser game launched in 2013 that invites players to guess the location of random Google Street View images. Casapu was already known as one of the top players in the world before he won the third annual GeoGussr World Championship in September.

At the beginning of the spring 2025 semester, School of Interactive Computing professor James Hays reached out to Casapu and invited him to collaborate on a new project. Hays was looking to create a dataset to evaluate VLMs' geolocation ability and reasoning. 

“VLMs are surprisingly good at geolocation right out of the box, even when they’re not trained to be good at it,” Hays said.

Hays and his colleagues, associate professors Alan Ritter and Wei Xu, took issue with many AI companies claiming that the VLMs they were releasing were not good at geolocation.

“When Open AI released GPT 4 Vision, there were privacy concerns about the model’s ability to geolocate someone based on photos they’ve shared on the internet,” Ritter said. “Open AI said this wasn’t a concern and claimed the model wasn’t good at geolocation beyond being able to recognize a city or famous monument. We found that wasn’t the case. These VLMs are state-of-the-art at image geolocation tasks.”

Show Your Work

Hays and Ritter enlisted a team of some of the world’s top geolocators. It consisted of Casapu, Joshua Diao, a master’s student in computer science, and Tejas Santanam, a Ph.D. student in industrial engineering. They each received 500 images to geolocate.

Team members recorded their reasons for each of their answers. The result was GeoRC, the first benchmark for VLM geolocation performance, consisting of 800 “ground truth” reasoning chains. 

Hays and Ritter gave the same images to GPT 5, Gemini, Llama, and Qwen. The highest-performing model geolocated with 90% accuracy — not far off from the team’s 96% score.

However, a major distinction showed up in the reasoning chains. While Casapu and Diao provided clear explanations for how they deduce each location, the VLMs either couldn’t provide reasoning for their guesses or were vague in their answers.

“The research community has been demanding explanations from these models,” Hays said. “For example, how do they know the location is in Italy?”

Hays has been researching this subject for almost 20 years. As a Ph.D. student at Carnegie Mellon University in 2008, he was the first researcher to take a machine learning approach to geolocation. He introduced a new algorithm that could estimate a geographic location from a single image.

“When experts have audited these reasoning chains, we’ve noted many suspicious or hallucinated attributes,” he said. “When they hallucinate a geographic property, why is it so often consistent with the correct guess?

“I believe they’re not revealing the true reasoning pathway that they used to determine the image was Italy. They’re just implicitly recognizing that it was Italy for many reasons, then hunting for evidence to support that. Some of the things they say are true and supported by the image, and some are fabrications.”

Practice Partner

Casapu said there may be only a handful of GeoGuessr players who can currently outperform some top-tier VLMs in geolocation, and it may not be long before no one can.

“I think it could be more difficult playing against these models than playing against another human because a human has the possibility of making mistakes at the top level,” Casapu said. “If a well-trained model has that level of consistency, that is far beyond a normal person, and it would be much more difficult to beat.”

He added that working with Hays and competing against a machine improved his skill level and provided valuable practice ahead of the world championship. 

“It helps to take a step back and see why you’re making the guesses that you are,” he said. “Since then, I’ve taken a more methodical approach to how I practice. Writing these things down is a great way to see what you know and see why you make the guesses that you do. It’s been a great training tool.”

Casapu will defend his title at the 2026 GeoGussr World Championship in September.

Hays, Ritter, Xu, Casapu, Diao, and Santanm are all co-authors of a paper on GeoRC along with lead author Mohit Talrej and Ph.D. students Ethan Mendes and Jim Thannikary. The paper will be presented next week at the 64th Annual Meeting of the Association for Computational Linguistics (ACL) in San Diego.