One in five women surveyed by Lean In.org and McKinsey & Company for their fourth annual study of women at work told the organization that they are often find themselves in meetings and other workplace situations where they are the only woman. That statistic doubles for women in senior-level positions, 40% of whom report being the lone woman. The data comes from a survey of 279 companies, with 64,000 employees participating.

“We’re inclined to say, ‘We’ve got a woman here at the table, so we must be good,’ not realizing there’s a cost associated with that for the woman sitting there trying to represent an entire 50% of the population,” said Alexis Krivkovich, a partner at McKinsey who led the report with Lean In.

This was the first time in its four years of conducting the survey that Lean In has asked this question, identifying the group of women it’s dubbed “the Onlys.”

Forty-five percent of women of color surveyed reported being the only person of their race in work situations, and 76% of LGBT women reported being the only one of their sexuality. Lean In didn’t include gender in that set of survey questions, so it did not provide statistics for how often these employees are the only women of color or LGBT woman.

Sign up: Click here to subscribe to the Broadsheet, Fortune’s daily newsletter on the world’s most powerful women.

Being the only one—on a team, in the C-suite, or on a project—has consequences. Women who spend time as “the Only” are one-and-a-half times more likely to think about leaving their jobs than women who work with other women. Eighty percent of women in the Only category, compared to two-thirds of women across the board, say they have experienced microaggressions, from being mistaken for someone more junior to needing to provide more evidence of their competence. Women who are Onlys also report being sexually harassed at higher numbers.

“These women are having a much worse experience in the workplace than women who have the opportunity to work with more women,” Lean In president Rachel Thomas, who co-founded the organization with Sheryl Sandberg, said. “They’re feeling like the odd woman out because they are.”

Onlys report feeling under pressure to perform and on guard at work. And here’s the twist: Of the 7% of men who say they are often the only man in the room at work, the majority say they feel included and “fortunate to be there.”

To solve this problem, Lean In recommends that companies hire and promote more women. But companies can also reconsider how they structure teams in the workplace and pay attention to teams where women are far outnumbered.

Across the board this year, Lean In found that women’s representation in corporate America hasn’t budged. The biggest problem is at that first promotion to the manager level, which women have trouble getting compared to their male peers.