Everyone Wants to Make the Earth Searchable. But AI Still Can't See It.
Why today's geospatial AI still struggles to understand the physical world.
Welcome to Ground Truth, OmniGeo’s publication on the intersection of geospatial AI and physical risk — for the people building in Earth observation and the underwriters, asset owners, and risk teams who have to trust them.
There's a race underway right now to do for the Earth what Google once did for the web: make it searchable. You type a question and get back an answer grounded in satellite imagery. The building blocks are everywhere, from Google's planet-scale embeddings to IBM and NASA's Prithvi, NVIDIA's Earth-scale models, and open foundation models like Clay and TerraMind multiplying by the month. The pitch is seductive: compress the planet into vectors, and anyone, an analyst, an app, an agent, can query the world in seconds.
I truly believe in where this is heading. But there's a problem the demos tend to skip over, and it isn't a small one. For all the talk of seeing the planet, AI doesn't actually see the Earth at all; it pattern-matches pixels, and we now have hard evidence of just how far that gap goes.
What the New EarthShift Benchmark Found
Last month, researchers at Arizona State released EarthShift, the first serious public benchmark built to measure not how well geospatial models perform, but how well they hold up when the world changes underneath them. They tested 8 leading foundation models, among them Clay, Prithvi, TerraMind, and DINOv3, across 11 tasks and five kinds of real-world "shift." The results point to some critical issues with these existing approaches.
Five ways the world shifts, and which ones break the models
Lead author Kelsey Doerksen summed it up in a single word: GFMs are “brittle.” Here are the five shifts that successfully unmoor a deployed model:
Sensor shift: Train a model on imagery from one satellite, then run it on another, say Sentinel-2 to Sentinel-1, or optical to radar. Both see the same ground through very different instruments, with different spectral bands and radiometry. The degradation from this type of shift was the most severe in the benchmark, with accuracy collapsing by up to 60%.
Geographic shift: Train in one part of the world and deploy somewhere else entirely. A field-mapping model trained on Germany’s large, orderly parcels falls apart over the smallholder plots of Cambodia, where field sizes, crops, and landscape look nothing alike. As you’d expect, the bigger the leap, the worse it gets.
Scale shift: This one is about resolution: the same scene at very different levels of detail. Picture a model trained on coarse public imagery, where each pixel covers ten meters of ground, then pointed at sharp commercial imagery measured in tens of centimeters. A rooftop that was a single fuzzy smudge becomes a richly textured surface, and the model’s grasp of object size and texture breaks down. Even systems with strong in-distribution scores lost a third to two-thirds of their performance.
Data source shift: This is the subtle one, and the most unsettling. Everything appears to match: the same task, sensor, resolution, and label classes. Only the provider or processing pipeline behind the images has quietly changed. Imagine two vendors both selling “analysis-ready” Sentinel-2 data, where one applies a different atmospheric correction, so pixels a human would swear are identical carry different values. The model still drops, which is what makes this case so dangerous: nothing looks wrong, and yet the answer is quietly off.
Temporal shift: Finally, the same place observed at a different moment in time. Think of a cornfield in June, lush and green, and then again in October after the harvest, reduced to bare soil and stubble. Same land, completely different pixels. This was the one shift the models mostly handled, which is itself the tell: they cope with the gentlest change and stumble on the four that define real deployment.
On average, these models shed around 20% of their performance the moment they were pushed beyond their training set. The real takeaway is more uncomfortable: models built specifically for Earth were no more robust than generic image models trained on ImageNet, and in several cases no better than a randomly initialized network. Domain-specific pretraining, the premise behind “large Earth models,” bought essentially no robustness, and fine-tuning didn’t fix it; sometimes it made things worse. As one GeoAI engineer put it, “models don’t fail loudly. they drift.”
And it isn’t only the models that see the Earth; it is also the ones you talk to. The whole promise of a searchable planet rests on vision-language models, and a second benchmark released in March, OmniEarth, ran those through 28 geospatial tasks and found they “still struggle with geospatially complex tasks," sometimes answering from language patterns rather than the image itself. A confident answer not grounded in the pixels is the textbook definition of geospatial hallucination.
Why the People with Real Money On the Line Aren't Running on Embeddings
When a decision is truly consequential, you don't see raw geo-embeddings making the call. There is always a human in the loop re-validating the output before anyone acts. That isn't mere conservatism, because the EarthShift results suggest it is the rational response. Much of what gets marketed as "Earth AI" is, underneath, a wrapper around models that don't generalize past their training distribution, and the industries with the most to lose feel that instinctively. As Melissa Rosa put it in a sharp essay on scaling Earth observation, a model can be accurate on average and still end up "confidently wrong in the case that matters most." Embeddings can tell you two places look similar, but similarity is not the same thing as attribution. Decision-makers need to know what actually changed, why it matters, how reliable the signal is, and what to do next, with calibrated confidence, much the way a 70% chance of rain is a number tested against millions of real outcomes.
What actually closes the gap
The path forward is not a bigger model. EarthShift found that scale doesn’t buy robustness: model size had no correlation with how a system held up out of distribution. The gap closes instead with three things the current wave mostly skips:
Measurement built for stability and comparability: utilizing a signal that means the same thing across different sensors, seasons, and geographies, rather than a representation that drifts the moment conditions change.
Attribution grounded in physical reality, not statistical similarity: answers tied to what is physically there and why it matters, validated against real ground truth, rather than a claim that “this vector is near that vector.”
Calibrated confidence: a system that tells you not only what it sees, but how much you should trust it under these specific conditions, improved through feedback loops between prediction and observed outcome.
These may seem like overly theoretical fixes, but over the coming weeks, we will dive into some of the technological innovations pioneered by our team at OmniGeo and how we believe we can meet these criteria in completely novel ways.
All of this is harder than building another foundation model; it demands fluency in both the underlying physics and the operational reality of the decision. But it is the only version an underwriter, an emergency manager, or a defense analyst can stand behind, and the only one that doesn’t end with an insurance customer losing their coverage based on a geospatial hallucination.
The web, after all, was made of links, and Google won by indexing them. The Earth is not made of links. It is made of atoms, and you cannot compress your way past that. The companies that define the next decade of GeoAI won't be the ones that make the Earth searchable. They'll be the ones that can hand you an answer you'd actually bet on when the stakes are real.
The measurement layer behind Ground Truth
OmniGeo’s first product, OmniFire, closes the earth measurement gap for insurers in wildfire-prone areas. OmniFire turns standard imagery into parcel-level measurements of what a property is actually made of, including roof material, defensible space, and fuel moisture. It’s vulnerability measured at scale and it’s live today. If you underwrite, price, or manage wildfire risk, visit www.omnigeo.ai/omnifire to learn more.
Want to learn more about the future of Earth AI? Visit us at OmniGeo.