In Florida, within the last month or so alone, Farmers Insurance announced that it was pulling out of the state’s market to “effectively manage risk exposure,” the company said, in a statement previously provided to Fortune. Shortly after, AAA said it would not renew a “very small percentage of higher exposure homeowner’s policies in Florida,” because of the state’s “challenging market,” according to a statement shared with Fortune previously.  

It’s becoming an increasingly difficult situation for the Florida and California housing markets, given that both experienced substantial increases in home prices during the pandemic. Not to mention that mortgage rates are back up, with the latest reading for the average 30-year fixed rate coming in at 7.37%, hitting a 20-year high. Affordability is shot, and insurance woes only exacerbate the problem. To such an extent, according to a John Burns Research and Consulting homebuilder survey, insurance concerns are somewhat slowing new home sales in Florida and California. 

John Burns Research and Consulting’s vice president of research and demographics, Eric Finnigan, told Fortune that they’d heard anecdotes from industry contacts that Florida and California’s distressed insurance industries were having an impact on their housing markets before they had data to back it up. It started with investors, he said, looking to buy properties and getting quoted twice or three times as much for coverage than a few years prior. In some cases, they couldn’t justify the cost and those deals fell through. With this survey, John Burns Research and Consulting asked builders how concerned buyers are about the availability and cost of homeowners insurance in their region, and the impact that concern is having on sales, Finnigan explained. 

“About a third of homebuilders in Florida are saying that buyers’ concerns about insurance availability and costs are somewhat slowing sales,” Finnigan told Fortune. 

Specifically, 32% of homebuilders in Florida are saying that buyers’ concerns over insurance are slowing sales, at least somewhat. But to be clear, in Florida’s case, 68% of builders suggest there’s no impact on sales. Still, in Northern California, 20% of homebuilders surveyed said that buyers’ concerns over property insurance are somewhat slowing sales, and in Southern California, 29% of builders said as much. To compare, on a national level, only 9% of builders said that insurance concerns were somewhat slowing sales, and in Texas, only 4% said the same. 

It’s clear that buyers in these markets are concerned. In Florida, 54% of builders said buyers are somewhat concerned over the availability and cost of homeowners insurance, and 14% said they were very concerned. And they have reason to be. Homeowners in Florida are already paying the highest insurance premiums in the nation, with an average premium of $6,000 per year, versus the U.S. average of $1,700 per year, according to Mark Friedlander, the Florida-based director of corporate communications for the Insurance Information Institute.

In both markets, insurance operating costs are up (although operating costs are up throughout most industries, Finnigan said). Additionally, there seem to be more and more extreme events, whether that’s extreme weather or natural disasters, resulting in homeowners filing insurance claims. That results in even higher costs for insurance companies, which is largely why we’re seeing insurers either pulling out of these states completely or renewing fewer policies. Those that are left can be more selective with who they cover, resulting in fewer options and high costs for homeowners. There are several factors at play, and these are just two of the many forces behind the insurance exodus.  

Interestingly enough, Finnigan said, “concerns about costs are just keeping people in rental units longer.” For someone who may have been considering buying a home, and already would have been stretching their budget to afford their monthly mortgage payment, these increases in property insurance are pushing that out of reach. 

Insurance woes may be a secondary concern, as Finnigan put it, but he still thinks the problem at hand is “serious.” Up to this point, he said, builders have been able to incentivize their buyers through mortgage-rate buydowns, for one. The longer rates stay at above 7%, the more insurance issues will magnify strained housing affordability in California and Florida. 

“As mortgage rates stay high, these markets have even more risks than, say, Georgia or Texas or Colorado, where these insurance concerns aren’t as salient,” Finnigan said, honing in on the fact that several would-be buyers are already trying to wrap their heads around monthly mortgage payments that have more than doubled in just a few years, and higher insurance costs only make owning a home more unaffordable. Maybe that means people continue to rent for longer, as Finnigan pointed out, or maybe it means a family that would’ve moved to Florida chooses not to, or delays their move, instead. 

“If these natural disasters, or extreme weather, get worse, I could see this intensifying and the impact actually getting worse,” Finnigan said. 

The big risk, in his view, is the impact insurance concerns will have on housing demand. For California, it could cause more people to move out of the state, and for Florida, it could cause fewer people to move there, Finnigan explained. As we head into the fall, we may see a seasonal slowdown in some markets and “if the insurance issue stays the way it is, or it gets worse, it’ll have an even bigger impact,” Finnigan said.