Today, editing is a manual, human process. Spelling and grammar checking software, such as the one built into Microsoft Word or the grammar checker Grammarly, can suggest when a word is incorrectly used or a comma misplaced. But it can’t recommend an entirely different way of writing a sentence to convey the same idea.
Now AI21 Labs, an Israeli startup cofounded by two well-known machine learning researchers and several veterans of an elite Israeli military intelligence unit, has debuted an A.I.-enabled editing tool that does exactly that. It can suggest variations of different lengths and tones, as well as help users find the best word to use in a particular circumstance.
“We are developing a true writing companion, a cowriter as opposed to a copy editor,” Ori Goshen, one of AI21’s cofounders, said.
Take the infamously inarticulate statement from President Trump, uttered when he was still a candidate during the 2016 presidential election campaign: “I know words. I have the best words.” The software AI21 Labs has created can suggest a range of ways to rewrite the sentence to capture the same intent, but using words that don’t even appear in the original. These include the more formal, “I am a master wordsmith and possess an impressive vocabulary,” as well as a version that’s simpler but still more articulate than the original, “I have a wonderfully rich vocabulary.” It can also suggest a more verbose phrasing: “I have an extensive vocabulary, and am skilled at choosing the most appropriate term to express myself in a given situation.”
The product, called Wordtune, is a free Google Chrome browser extension that can edit sentences in programs like Google Docs, Gmail, and Sheets, as well as within social media sites like Facebook and Twitter. Wordtune does not yet work with Microsoft Word, but the company hopes to integrate it with the well-known word-processing software sometime soon.
Goshen is a serial entrepreneur as well as a veteran of the Israeli Defense Forces’ Unit 8200, the main signals intelligence division of the Israeli military. Several other veterans of the unit also work for the company, he said.
He cofounded AI21 Labs in 2017 with two of Israel’s best known A.I. experts: Yoav Shoham, a former Stanford University artificial intelligence researcher who sold two previous startups to Google and worked for the search giant for two years, and Amnon Shashua, a computer scientist at Hebrew University who is also the chief executive at Mobileye, a subsidiary of Intel that makes artificial intelligence software for self-driving cars and the automotive industry. Shai Shalev-Shwartz, another well-known Israeli machine learning researcher at Hebrew University, also does work for the company.
Shoham said that the company’s lofty goal is to use A.I. “to fundamentally transform how we read and write.”
Wordtune is an example of the rapid advances in natural language processing—a kind of A.I. that can manipulate and to some extent “understand” language—that have occurred in the past two years. These advances have been driven by two developments: a new kind of design for a neural network, a kind of machine learning loosely based on the brain, called a Transformer, and the advent of massive language models that take in tens of billions of variables and are trained on colossal amounts of text.
AI21 Labs is launching Wordtune into an increasingly crowded field of competing writing tools that can automatically compose or rephrase text. A competing system called Quillbot, built by a Chicago startup, suggests paraphrasing and can rewrite whole paragraphs. Google is increasingly incorporating more sophisticated auto-complete functions into its products, including Gmail. Microsoft has launched a feature in Microsoft Word that uses A.I. to suggest alternate ways to phrase a sentence. OpenAI, which has created one of the largest language models, called GPT-3, has licensed the system to a startup called OthersideAI, which uses the software to automatically generate emails from a few bullet points.
Unlike GPT-3, AI21 Labs’s system is a fusion between neural network–based language models and an older form of artificial intelligence that seeks to represent human knowledge, like vocabulary and the meaning of words, in a graph structure, Shoham said. This fusion is part of what enables Wordtune to have a much better understanding of the concepts being expressed in a piece of writing than some of the pure statistical approaches based on ultra-large language models.
The company has also made some fundamental research breakthroughs in the way large language models work. One, which it published earlier this year, is an improvement on a large language model invented by Google called BERT. BERT is based on hiding random words in a large text and then asking an A.I. system to predict the hidden words. AI21 Labs refined this system, creating an A.I. language model called SenseBERT, in which the masking is not completely random but is weighted toward the parts of the sentence that are most important for conveying meaning. “This allows it to capture the true sense of a sentence,” Shoham said.
As a research project, the company also built a system, which it calls Haimke, that can, like the system built by OthersideAI, turn bullet points into prose. Another system, called Haim, works a bit like GPT-3, composing long coherent passages from a few sentences of human writing. Haim differs from GPT-3, though, in that the human using it can help guide the text the A.I. creates by writing a series of “waypoint” sentences. The software fills in the gaps between these waypoints with novel prose, but the waypoints serve as logical guideposts, so the software is less likely to “run off the rails” and generate nonsensical prose, Shoham said. Neither system has yet been turned into a commercial product.
