If you had to guess why people who work in the gig economy would prefer a regular full-time job, finances might come to mind first. Employees, after all, are usually entitled to lots of benefits, including paid time off, that gig economy workers, or freelancers, don’t get. A predictable paycheck is nice, too.

But for about half of independent workers — especially Millennials, who make up about one-third of freelancers in the U.S. — the main drawback is something else entirely: Feeling like an outsider.

Deloitte recently polled about 4,000 U.S. workers consisting, like the workforce overall, of a mix of full-time employees, part-timers, and independent contractors. As you might expect, about 60% of people with full-time jobs said they’d hesitate to strike out on their own because of “inconsistent cash flow,” and 42% cited losing company-paid benefits as a main concern.

The loneliness factor

But among those who were either past or current freelancers, a different pattern emerged. Almost 70% said that, given the choice, they wouldn’t opt for self-employment in the future. Half of the Millennial freelancers surveyed said that “a lack of connection to a company’s internal culture” discouraged them, a higher percentage than for any other generation. That feeling of isolation, even loneliness, was also a big reason why the survey found that fewer than half (48%) of past and present freelancers (of all ages) reported being “very satisfied” with the experience.

What employers can do

Clearly, contractors who don’t work side by side with regular employees miss out on the kind of informal day-to-day contact — coffeepot chitchat, last-minute lunch meetings, impromptu softball games — that makes people feel they belong. So, says Deloitte’s chief talent officer Mike Preston, companies need to find other ways of drawing freelancers into the fold, including offering training and coaching to help develop their careers.

“We’re working now to create a culture to attract and retain contingent workers,” he says. Because “community is important,” he adds, Deloitte has started an online portal where contractors can find postings on open project opportunities inside the firm, along with other information that’s also available to regular employees.

Making freelancers feel they’re a part of a client company’s culture may matter even more as time goes on. By 2020, according to new research, independent contractors will number 54 million in the U.S. (up from 40 million now), or some 45% of the private-sector workforce, and many have “unique skill sets and deep experience that can be valuable,” notes Preston. He sees a competitive advantage ahead for employers who can “shape an environment that creates an affinity — whether the person is working with them on one project or for a lifetime.”