The U.S. tech sector was broadly opposed to the candidacy of Donald Trump, motivated by concerns from data privacy to telecom regulation. But the biggest concern may have been the potential impact of Trump’s immigration stance on tech staffing.

Trump essentially made his political name by broadly opposing immigration, from illegal migrants to Syrian refugees. But he has sent mixed messages about the issue most relevant for tech companies, the H-1B visa program for temporary foreign workers. The majority of H-1B workers are in technology fields, and the tech sector says it needs more workers than the program currently provides because there aren’t enough skilled Americans to fill those jobs.

Get Data Sheet, Fortune’s technology newsletter.

But President Elect Trump has characterized the program as a way for companies to undercut American wages with cheap foreign labor. He proposed that the program be restricted, that companies be required to hire Americans first, and that the prevailing wage for H-1B workers be raised.

Trump has since seemed to reverse that position, saying that “we need highly skilled people in this country, and if we can’t do it, we’ll get them in.” But he could still restrict immigration on a country-by-country basis, and is now staffing his cabinet with figures who oppose immigration in all forms. That includes Attorney General candidate Jeff Sessions, who argues that legal immigration primarily benefits economic elites at the expense of workers, but has also made racist statements.

Trump has also threatened to end or renegotiate America’s NAFTA trade agreement with Canada and Mexico, which created another visa path for professionals. And he is seen as unlikely to move forward with a special immigration rule for foreign-born entrepreneurs, who founded almost 45% of startups in 2012.

That mix of threat and uncertainty has Silicon Valley on tenterhooks about what happens next. Speaking with Bloomberg, Y Combinator co-founder Paul Graham said Silicon Valley would be “the first to suffer if it got harder to come to America or if increasing xenophobia made fewer people want to.”

The consequences could come in myriad forms, not only by limiting staffing efforts, but because those foreign workers would, by necessity, go to the competition in other countries. Indeed, China is already inviting talent who may be blocked from the United States to consider its tech sector. Not to mention that the U.S. tech industry already struggles with diversity even with the H-1B visa program and current immigration policies in place.

For more on Trump and tech, watch our video.

Shifting perceptions of American openness to foreign workers may have as much impact as any regulatory change. Hate crimes and racist incidents have surged following Trump’s victory, which was also seen as a win for the new wave of “alt-right” groups. Those groups lobbied hard for Trump, and advocate for white identity politics.

A huge proportion of international workers joining U.S. tech companies are from India. As immigration lawyer Ava Benach put it to Bloomberg, “people who have brown skin might feel pretty uncomfortable coming to the U.S. under these circumstances.”

If they can’t get the workers they need, tech companies’ growth could be severely throttled. And with tech a major ingredient in U.S. job and GDP growth, the consequences could reach well beyond the Valley.