Allowing staff in entry-level roles to use A.I. tools like ChatGPT to help them with their work brings about big productivity boosts, according to new research.
In a new study, researchers from Massachusetts Institute of Technology and Stanford University analyzed the impact generative A.I. tools like ChatGPT had on productivity at an unnamed Fortune 500 software firm.
Generative A.I. models are programmed to use what they have learned from examples they have been shown in the past and generate something completely new based on that information.
Using data from 5,179 customer support agents, the research team found that workers who had access to an A.I.-based conversational assistant were 13.8% more productive than those who did not.
Productivity was measured by how many issues individual agents resolved per hour.
“This increase reflects shifts in three components of productivity: a decline in the time it takes an agent to handle an individual chat, an increase in the number of chats that an agent is able to handle per hour (agents may handle multiple calls at once), and a small increase in the share of chats that are successfully resolved,” the study’s authors wrote in their paper, which was published by the National Bureau of Economic Research.
The greatest productivity boost was seen among novice and low-skilled workers, according to the research paper, while access to generative A.I. had “minimal impact” on experienced and highly skilled workers.
In less experienced staff, A.I. helped them work 35% faster than they had without the tech’s assistance.
“Treated agents with two months of tenure perform just as well as untreated agents with over six months of tenure,” the research team said, arguing that there was evidence generative A.I. models “disseminate the potentially tacit knowledge of more able workers and help newer workers move down the experience curve.”
They also argued that their study showed A.I. assistance “improves customer sentiment, reduces requests for managerial intervention, and improves employee retention.”
“Our overall findings demonstrate that generative A.I. working alongside humans can have a significant positive impact on the productivity and retention of individual workers,” the researchers wrote.
They said that “to the best of our knowledge,” their research was the first time the impact of generative A.I. on workplace productivity had ever been investigated in a real-world setting.
Since OpenAI’s chatbot ChatGPT became a cultural phenomenon in recent months, the role of artificial intelligence in the workplace has become a hot topic of debate.
While some workers are concerned that the technology could displace them from their roles, many big-name CEOs, such as IBM boss Arvind Krishna, have said employees should instead be focused on working “hand in hand with artificial intelligence.”
Walmart’s chief people officer Donna Morris wrote in a recent op-ed for Fortune that tech like A.I. was actually “empowering our people,” while Bill Gates has prophesized that in years to come, everyone will have their own “white collar” virtual assistant.
‘1,000% a game changer’
Alex Galtsev, founder of Realiste—a Dubai-based A.I.-powered real estate platform—told Fortune that the productivity boost observed in the MIT/Stanford study “is truly significant and should not be underestimated.”
“It’s crucial to recognize that even small improvements in productivity can have a substantial impact on business operations, leading to increased efficiency and profitability,” he said in an email. “We have witnessed a significant increase in productivity in our organization thanks to the incorporation of A.I. into our work processes. Our investment managers’ productivity has increased by up to 200% by utilizing artificial intelligence to identify the best properties on the market for our clients. I strongly believe that this level of improvement is achievable for many other businesses as well, given the right approach and tools.”
Galtsev said he believed there would be increased adoption and innovation in A.I. over the next two to three years, arguing that the technology would become more user-friendly and thus easier to integrate into business operations.
“By automating repetitive tasks, streamlining decision-making processes, and providing unique insights, these tools can offer productivity improvements that are difficult or impossible to achieve through traditional means,” he said.
Meanwhile, Jack Hampson, CEO and founder of British A.I. consultancy Deeper Insights, said his company and its clients were seeing much bigger productivity gains than the 14% recorded in the study.
He told Fortune in a phone call on Monday that A.I. tools like ChatGPT will “1,000%” be a game changer in the workplace.
An HR firm Deeper Insights works with has seen productivity boosts of 20% to 40% since it started using a ChatGPT-like large language model to assist with tasks like writing policy, retrieving policy information, and searching for employment case law updates, Hampson told Fortune.
Deeper Insights itself is also using tools like ChatGPT to get work done much faster than it could be completed manually, he added.
“We’re obviously very data hungry, and a lot of the time we don’t have enough data from a client to train an [A.I.] model,” Hampson explained. “So we’re now using things like ChatGPT and other large language models to create alternative training datasets, and that is saving us a lot of time and money. In a three-week or a three-month project, we probably spend about two to three weeks on the creation of training datasets, and that time is now a day or two.”
The authors of the generative A.I. study stressed in their paper that their research was “not designed to shed light on the aggregate employment or wage effects of generative A.I. tools.”
“Our results do not capture potential longer-term impacts on skill demand, job design, wages, or customer demand,” they said, but they noted that their findings did raise questions about whether—and how—workers should be compensated for the data they provide to A.I. systems, adding that the productivity boost was likely “driven by the A.I. system’s ability to embody the best practices of high-skill workers in [the] firm.”
