A federal housing voucher allowed Daniels to escape her crime-plagued neighborhood in Dallas, Texas, and move her four children to the suburb of Garland.

That move helped Daniels find a good job, working for a property management company. Today, she cherishes the small, two-bedroom house she rents, with a yard where her children can safely play, away from drugs and gang violence in Dallas.

But if Daniels, 31, were to lose the $1,082 monthly stipend she receives, she has no doubt what would happen. “I would have to move back,” she said. “I wouldn’t be able to afford it.”

Pulling Americans out of poverty is a subject close to Ryan’s heart. And President Donald Trump has frequently talked about aiding the inner cities

Rhode Island Protesters Call Paul Ryan a ‘Coward’ with ‘Wall Street Values’.

But the housing assistance provided by the U.S. Department of Housing and Urban Development, or HUD, could fall victim to fiscal policies under consideration by the White House and Congress, which include a massive tax cut and increased military spending, according to a dozen congressional aides who spoke to Reuters.

While the White House has not been specific about its plans for HUD — the budget process remains in flux — it has called for a $54 billion cut in non-military discretionary domestic programs in the next fiscal year, which likely will dramatically impact safety-net programs that are not entitlements such as Medicare and Social Security, budget experts said.

Altogether, housing advocates paint a bleak picture of the landscape for low-income housing under the Trump administration. They warn that deep cuts to housing funds would force some people out of their homes and hollow out grant programs meant to revitalize urban neighborhoods.

There is also a ripple effect, they warn: Rental prices would likely rise alongside declines in affordable housing projects and vouchers available to renters. That would make it tougher for renters to save enough to eventually buy homes.

“It’s a bad cycle,” said Carol Galante, a former top HUD official during the Obama administration. “It puts pressure on the rental market. The rents are higher and higher so people can save less and less.”

Trump’s Border Tax: Here’s Number the Pundits Are Missing

HUD provides about 5 million Americans with some form of housing assistance, either through vouchers to renters, subsidies to landlords or public housing projects, which comprises about 85% of its budget. It also sends about $8 billion annually directly to communities through grants.

Even so, only about one-quarter of those eligible for assistance in the country receive it.

Housing advocates say changes to its budget or mandate would be directly felt in low-income communities. They point out that in 2013, when the legislatively mandated budget cuts known as sequestration hit HUD, more than 100,000 renters nationwide lost their housing support.

Douglas Rice, an expert at the Center for Budget and Policy Priorities, a nonpartisan think tank in Washington, estimates that for every one percent cut in HUD’s budget, 20,000 renters would lose their assistance

But critics of the agency, including some Republican lawmakers, say its anti-poverty and community-development programs are inefficient and wasteful, and that it has failed to live up to its mission. Today, the proportion of Americans who live below the poverty line — 14.5% — is about the same as it was 35 years ago.

Russian Real Estate Deals Never Materialized for Trump

HUD has also at times had difficulty shaking the image of a bureaucratic agency that is vulnerable to corruption. It was at the center of a bid-rigging scandal during the Reagan Administration. And Alphonso Jackson, the HUD secretary under President George W. Bush, resigned in 2008 amid allegations that he steered contracts to friends. Charges were never brought against Jackson.

“There are some sports teams that never seem to gel. I think we’re a little like that at HUD,” said Bud Albright, a Washington lobbyist who worked at the agency when Republican Jack Kemp was secretary in the early 1990s.

Trump, who has not been specific on his plans for the agency, selected Ben Carson to lead HUD. The former neurosurgeon and Republican presidential candidate won Senate confirmation last week.

Although Carson’s views hew closely to Republican orthodoxy on how too much government can discourage people from working hard, his upbringing in inner-city Detroit gives him a unique perspective: his mother received food stamps to provide for her family and he was raised around housing assistance programs similar to those he will now manage.

Carson, who declined requests for interviews by Reuters, has pledged to fight to protect HUD’s housing-assistance budget and preserve the agency’s community-development initiatives. He also said he would push to include funding to rehabilitate public-housing facilities in Trump’s proposed $1 trillion infrastructure plan.

How Donald Trump Just Raised Many Mortgage Bills

But given the White House’s determination to cut domestic spending, Carson will have to fight for every dollar. A HUD spokesman declined to comment on which programs could be hurt by any possible budget cuts. The White House’s Office of Management and Budget did not respond to a request for comment.

Congress, too, is looking to narrow HUD’s reach.

Ryan has called for work requirements and time limits on those who receive housing assistance, similar to how child support, food assistance and other welfare benefits function.

Tax reform spearheaded by Ryan and Kevin Brady, the House Ways and Means chairman, could ultimately do away with a key tax credit used by developers to build affordable housing, or could drastically curtail the credit’s use.

One vocal critic of HUD is Jeb Hensarling, Republican chair of the House Financial Services Committee, which oversees the housing agency. Hensarling plans to introduce legislation this year to narrow the $1 trillion portfolio of the Federal Housing Administration, which helps low-income and first-time homebuyers purchase homes, his office said.

Who Knew This City Had Some of the World’s Most Unaffordable Housing?

Hensarling has said he fears that if home values drop the FHA would require another federal bailout as it did in 2013, when it received a taxpayer-funded infusion of $1.7 billion to cover its losses.

If the cuts to HUD’s budget are as severe as some expect, those who are on waiting lists now for vouchers will be staying on them for a very long time, and recipients such as Shiva Daniels, who has been receiving assistance for six years, will be at risk of seeing that support end.

That would mean losing her small house, with the yard she sees as a safe haven for her children. “It might not feel like it’s a big deal, but it is,” Daniels said. “When you feel good, it allows you to do better, and do better for them.”