Why businesses are buzzing over transformers

The effort received a huge lift from the introduction of transformers, a concept developed by the Google Brain research lab that helped to improve natural language processing. Transformers involve a variety of neural networks, the software used for deep learning, that excel at teaching computers language skills.

Because of how they encode data, transformers are particularly good at discovering relationships between words and phrases in long sequences that more conventional NLP software would overlook. As a result, companies like Google, Facebook, and the A.I. research firm OpenAI have created powerful language systems that can produce seemingly humanlike text based on written prompts. 

Now, Aidan Gomez, a former member of the Google Brain team that developed transformers, aims to make a business out of selling transformer-based language technology services to companies. His startup, Cohere, said Tuesday that it had landed $125 million in funding, led by Tiger Global and other investors including Radical Ventures, Index Ventures, and Section 32.

Gomez will face stiff competition from cloud giants like Google and Amazon, which sell similar software. Additionally, OpenAI, with help from a high-profile partnership with Microsoft, is also trying to commercialize its language software.

What distinguishes Cohere from others, Gomez says, is that the startup is focusing on developers who need easy-to-use language tools that can be deployed for more than one task, like analyzing what the public is saying about a certain topic on social media, a common use of NLP by companies. Most available language tools from rivals generally do only one task well, he explains.

Gomez also says that companies can bring their own language data tailored to their industry so that Cohere can help them build NLP software that recognizes the unique jargon and idiosyncrasies of specialized markets like health care. Although big cloud companies including Google and Microsoft will likely work on custom NLP projects for large customers like financial services giants, Gomez claims that they don’t yet offer easy-to-use developer tools for smaller companies that provide similar capabilities.

“For all the startups, the students, like the wide swath of developers that exists—the long tail—they’ll never get that sort of treatment,” Gomez says.

Cohere has built its own transformer-based language systems that are comparable to popular language software like Google’s BERT and OpenAI’s GPT family of software, Gomez says. Tech firms are increasingly building larger versions of these language models that include hundreds of billions of parameters, under the assumption that the bigger they get, the better they’ll understand language.

Gomez declines to comment about the size of Cohere’s language models, however, saying that size doesn’t accurately predict how well they’ll perform in everyday business tasks.

“These models cost millions and millions to train, and we just keep increasing [their size].” Gomez says. “Getting into a ‘largest model battle’ isn’t a productive direction going forward for the field.”

Jonathan Vanian 
@JonathanVanian
jonathan.vanian@fortune.com

A.I. IN THE NEWS

Texas to take Facebook parent Meta to court. The state of Texas filed a lawsuit against Facebook parent Meta that alleges the social media giant illegally collected biometric data of users and engaged in deceptive business practices. The lawsuit is similar to a previous lawsuit between Facebook and Illinois that Meta settled in 2021 for $650 million.

YouTube gets a boost of A.I. YouTube is now using a variant of the A.I. technology developed by Alphabet’s DeepMind A.I. unit to improve the quality of video compression so that videos stream more efficiently, Fortune’s Jeremy Kahn reported. The A.I. software was originally created to help computers master games like Chess, Go, and old-school Atari video games.

There’s gold in the A.I. hills. The startup KoBold Metals has raised $192.5 million in a new funding round, according to a report in the Wall Street Journal that cited unnamed sources. The startup’s technology uses A.I. techniques to help guide companies in drilling for minerals, such as those used for developing batteries for electric vehicles. KoBold Metals is backed by several high-profile investors including Bill Gates’ Breakthrough Energy Ventures, investor and technologist Sam Altman’s Apollo Projects, investor Mary Meeker’s Bond Capital, and mining company BHP Group.

A.I. salaries come down. Salaries for machine learning, natural language processing, and A.I. experts fell in 2021, declining by 2.1%, 7.8%, and 8.9%, respectively, according to a report by IEEE Spectrum that cites data from the recruitment firm Dice. In 2021, the average yearly salary for a machine learning expert is now $122,597, an NLP expert $121,286, and an A.I. generalist $120,168. A Dice representative told the publication that A.I. salaries could be declining as more technologists learn and master A.I. technologies, thus alleviating a shortage of talent. 

EYE ON A.I. TALENT

Cohere told Fortune that it had hired Bill MacCartney to be the machine learning startup’s vice president of engineering and machine learning. MacCartney was previously the director of proactive intelligence at Apple and a managing director at BlackRock.

Tenstorrent has picked Matthew Mattina to be the A.I. computer chip startup’s vice president of machine learning. Mattina was previously a distinguished engineer and senior director of machine learning research at semiconductor design giant Arm.

Asapp has hired Priya Vijayarajendran to be the contact center technology company’s chief technology officer, the tech news service Protocol reported. Vijayarajendran was previously a Microsoft vice president of data and A.I.

EYE ON A.I. RESEARCH

A.I. is more than fun and games. Researchers from Sony’s A.I. team published a paper in Nature detailing how they used deep reinforcement learning to teach computers how to drive race cars in the video game Gran Turismo Sport. Through the process of trial and error, Sony’s A.I. became proficient enough to beat some of the leading human Gran Turismo Sport video game players during a tournament.

The paper marks another example of how technology companies like Google, Facebook, and now Sony are creating cutting-edge A.I. that is besting human players at increasingly difficult games. Although these A.I. examples are a far cry from humanlike intelligence, they are far more sophisticated than older software and can perhaps be used for more practical uses involving planning and strategizing.

FORTUNE ON A.I.

Tyson Foods CEO is betting on A.I. and automation to reduce labor costs—By Sheryl Estrada

Europe is terrified of semiconductor irrelevance. Now its tech champion is calling for a massive new alliance—By Christiaan Hetzner 

The CIA has been conducting mass surveillance in the U.S. with minimal oversight—and the program’s uncovering is bad news for Big Tech—By  David Meyer

Activision Blizzard was their dream job. The workplace was a nightmare—By Courtney Rubin

BRAIN FOOD

The human cost of data labeling. The data-labeling company Sama and its work with Facebook is the subject of an extensive report in Time that details the intense working conditions of African labelers in Kenya who “perform the brutal task of viewing and removing illegal or banned content from Facebook before it is seen by the average user.”

Sama pitches itself as an “ethical AI” outsourcing company, the article said. The company maintains a global workforce that performs tasks like data labeling so that companies can improve their machine learning systems. Researchers have previously called this kind of data labeling practice “ghost work” because little is known about the working conditions of this global labor force that helps companies develop A.I.

The article describes how Facebook is outsourcing its content moderation operations in countries with cheap labor, which “has led some observers to raise concerns that Facebook is profiting from exporting trauma along old colonial axes of power, away from the U.S. and Europe and toward the developing world.”