This week two different stories hammered home the extent to which today’s generative AI startups have adopted a “move fast and break sh-t” approach and how this may come back to haunt them—and us.
The first story is that the Recording Industry Association of America (RIAA), which represents major music labels, including Sony, Warner Music, and Universal Music, filed copyright infringement lawsuits against two well-funded AI startups, Suno AI and Uncharted Labs, which both offer text-to-music genAI models. (Suno’s is eponymous, while Uncharted’s is called Udio.) The complaints in both suits contain a number of pretty damning examples in which the RIAA demonstrates that Suno and Udio can, when prompted with an artist’s name—or even just a simple description of the sort of vibe that artist is known for—and also given the lyrics to one of that artist’s songs, produce works that substantially overlap the original, copyrighted song in composition and rhythm.
Both companies have been evasive about what data they used to train their AI models. Unchartered has said that Udio is trained on “a large amount of publicly available and high-quality music” that was “obtained from the internet.” Suno CEO Mikey Shulman has said the company uses a “mixture of public and proprietary data” and, according to the RIAA’s complaint, told the association’s lawyers only that its training data was “confidential” while not denying the RIAA’s contention that it had hoovered up vast libraries of copyrighted songs. Shulman said in a statement issued in response to the RIAA suit that its AI model produced “transformative” works and noted that the company does not allow users to prompt the model with an artist’s name. (The RIAA showed that this guardrail was easily overcome by simply adding spaces between the letters in a name.) Shulman also said that rather than engaging in a “good faith discussion, [the record labels] have reverted to their old lawyer-led playbook.”
The problem for Shulman and his counterparts at Udio: That old lawyer-led playbook works. The RIAA has deep pockets and a strong track record of winning similar lawsuits against companies that also claimed that somehow copyright law didn’t apply to a novel technology. Remember, it was the RIAA that essentially drove Napster out of business in the early 2000s, and it eventually forced Spotify to strike a licensing and royalty deal for the music streamed across its platform. Those lawsuits essentially established the ground rules of a new technological age. These ones will likely help do the same for the genAI era.
While it is possible that a judge will rule that using copyrighted material to train an AI system is, as Suno and other AI companies such as OpenAI and Stability AI have contended, “fair use,” it is highly unlikely any court will rule that creating outputs that substantially plagiarize the original, copyrighted works does not constitute infringement.
The troubling thing is that it has been obvious to anyone who used Suno’s or Udio’s text-to-music generators that they were trained on copyrighted material and could generate infringing outputs. And yet the most prestigious venture capital firms and investors in Silicon Valley have chosen to reward this “better to beg forgiveness than ask permission” business culture by doling out huge amounts of money to these ethically challenged startups. In May, Suno announced a $125 million venture capital round led by Lightspeed Venture Partners, Founder Collective, Matrix, and Nat Friedman and Daniel Gross.
The core ethical principle here is consent. It is a point that Neil Turkewitz, a former RIAA executive who has emerged as one of the most passionate voices railing against tech companies hoovering up data without permission to train AI models, has made repeatedly. If generative AI is the platform of the future, do we want it to be built on a foundation of theft and nonconsensual use? Or would we rather put consent at the heart of the business models on which the AI age is being constructed?
The other AI news item that caught my attention this week also revolves around consent. An investigation by Wired found that popular genAI search engine Perplexity was almost certainly violating its own pledge to respect publishers’ wishes to prevent AI companies from scraping their content for free. The code, called robots.txt, is a way of asking bots used by specific tech companies not to scrape the data found on the page. The protocol is voluntary—just a courtesy, really—not something legally binding. But the point is that Perplexity said it would abide by the standard, and then hasn’t. In fact, Wired caught it using an IP address to troll websites that it had not published as belonging to one of its web-scraping bots—which certainly makes it seem like Perplexity knows exactly what it is doing. Perplexity told Wired in a statement that the publication’s questions “reflect a deep and fundamental misunderstanding of how Perplexity and the internet work,” but then it provided no further explanation of how, exactly, Perplexity does work.
This is in some ways even worse than what Suno and Udio have done. The AI music companies at least had the decency to stare awkwardly at their feet and mumble when confronted about taking copyrighted works without consent. Perplexity is looking us in the eye and saying, “of course, we won’t scrape your data (psych!).” Wired points out that summarizing news stories, which is what Perplexity does, is legally protected. (The story points out some other issues with Perplexity too, such as the fact its AI models sometimes summarize news stories incorrectly, and other times the summaries basically plagiarize the original work, which could be a copyright violation in some circumstances.) But I am more interested in the issue of ethics. And here, consent is key.
The corollary to Trukewtiz’s question is what happens if we don’t put consent at the heart of our data gathering practices? Well, what happens is that we all—as consumers or customers of this technology—become complicit. Silicon Valley’s louche ethics become ours.
This collective complicity is why we should all insist on tech companies doing better—and since they are unlikely to clean up their act unless forced to do so, why we need regulation. The EU’s AI Act, which requires that companies disclose if they have used any copyrighted material in training, is a start. The U.S. should consider similar rules.
But we should also insist as consumers and customers that tech companies come clean about what they are doing. We should also require that companies certify that they have consent to use the data their AI models are ingesting. A few weeks ago in London, I met with Ed Newton-Rex, a composer and entrepreneur who for a time headed Stability AI’s music generation efforts, before resigning over the company’s approach to consent and copyright. Newton-Rex has launched a new venture, Fairly Trained, that seeks to be a kind of Good Housekeeping Seal of Approval for AI companies’ data procurement practices. So far, though, only about 14 companies’ AI models have been certified. More work is clearly needed.
With that, here’s more AI news.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
Correction, June 27: The news item below on EvolutionaryScale’s fundraise has been corrected to reflect that Amazon participated in the funding round, not AWS.
Before we get to the news…Join me and Fortune as we delve into AI’s role in reshaping business operations and tackling challenges unique to Asian markets. We’ll be discussing AI’s impact on the bottom line and on everything from geopolitics to the environment at Fortune Brainstorm AI Singapore, July 30-31. Enjoy two days of dynamic interviews, fast-paced panels, live demos, and meaningful networking opportunities at the Singapore Ritz Carlton. Don’t miss out—reserve your spot today to stay ahead in the new race for AI. And for readers of Eye on AI, I can offer you a special discount code, good for 50% off the normal delegate’s price. Just use the code BAI50JeremyK when you register here.
**Finally, a reminder to preorder my forthcoming book Mastering AI: A Survival Guide to Our Superpowered Future. It is being published by Simon & Schuster in the U.S. on July 9 and in the U.K. by Bedford Square Publishers on Aug. 1. You can preorder the U.S. edition here and the U.K. edition here.
AI IN THE NEWS
Nvidia shares, darling of investors trying to play the AI boom, fall sharply. The chip design company that sits at the heart of the AI boom has seen almost $500 billion wiped off its market valuation in just three days, the largest ever loss for any company over such a short period. The reason for the decline is unclear, my Fortune colleague Christiaan Hetzner reports, although market watchers speculate that it is profit-taking ahead of Nvidia’s quarterly earnings announcement tomorrow, when it is possible the company might fall short of sky-high expectations. There is also mounting concern that data center spending may not expand as fast as some investors anticipate and in recent days a number of investment banks published reports noting just how insanely high Nvidia’s valuation had become, which might have convinced some investors that it was a good time to bail out of the stock.
Apple is reportedly in talks with Meta to use its open AI models. That’s according to The Wall Street Journal, citing sources familiar with the discussions. Meta’s open models, which Apple could run on its own servers, might help the company ensure it can protect user privacy and also refine Meta’s AI models for its own purposes. Under a deal with OpenAI announced earlier this month, Apple agreed to pass some Siri queries that its own AI models can’t answer to OpenAI’s ChatGPT 4-o, if the user consents. But some privacy advocates have criticized Apple for this arrangement. And in that case, Apple cannot as easily modify OpenAI’s models to meet a specific niche demand. On the other hand, it's more difficult to build guardrails around open AI models such as Meta’s, which may open up other security concerns. The Journal reported that Apple has also held talks with OpenAI rival Anthropic and AI search engine company Perplexity.
AI deepfakes being used for political manipulation are the most common form of genAI misuse. That’s according to a study from Google DeepMind and Google unit Jigsaw, which works to combat threats to open societies. The results were reported in the Financial Times. The research found that using AI to create realistic but fake images, video, and audio of well-known people was almost twice as common as the next highest misuse of generative AI tools, which was to produce text-based misinformation. In both cases, political influence or persuasion was the primary motive for generating the false content. The second most common motive for misusing generative AI was to make money, either by charging people for tools that let them create deepfakes, including nonconsensual pornography, or by using genAI to compose phishing emails or false identity documents to use in other kinds of financial scams.
Amazon is secretly preparing an LLM to rival OpenAI’s ChatGPT and power a new Alexa. That’s according to an article in Business Insider, which quotes Amazon insiders familiar with the project as well as an internal company the publication says it obtained. Amazon CEO Andy Jassy is personally monitoring the project, which is codenamed “Metis,” in reference to the Greek goddess of wisdom. Metis, which is being powered by an Amazon LLM called Olympus, is supposed to be able to hold a dialogue like ChatGPT, but according to Business Insider, uses retrieval augmented generation (RAG) to try to make its answers more factually accurate and up-to-date than those that other LLMs provide. The publication says Amazon is also trying to give Metis the ability to make API calls and use software tools, giving it some “agentic” characteristics. It's believed Metis will be used to update Amazon’s Alexa digital assistant. As my colleague Sharon Goldman reported in Fortune last week in a must-read piece, Amazon’s Alexa unit has been struggling for both technical and bureaucratic reasons to update Alexa for the generative AI era.
“Meta AI Mafia” members grab $142 million seed investment round for their AI biotech startup and launch a massive new LLM for biology. That’s according to a story also by my Fortune colleague Sharon Goldman. The new company, founded by Alexander Rives, who led Meta AI’s “protein team” before the company disbanded it amid big job cuts last year, along with other former members of that team, is called EvolutionaryScale. It has secured a massive new $142 million funding round from Silicon Valley investors Nat Friedman and Daniel Gross, and Lux Capital, with participation from Amazon, NVentures (NVIDIA’s venture capital arm), and angel investors. It needs that money because it's training huge LLM-based models designed to allow scientists to specify characteristics of a protein and then get a DNA recipe for making a protein that will perform exactly that function. EvolutionaryScale also announced it was releasing the first of these models, called ESM3, today.
EYE ON AI RESEARCH
Data indicate that freelancers really are losing their jobs to AI. I'm one of those people who believes fears of AI-induced mass unemployment are overblown. Most people’s jobs involve too many disparate, complex tasks and workflows to be easily automated away by AI. AI will change our jobs, but not eliminate them entirely. Of course, one counterpoint to that argument is that these days, a lot of people in the economy are freelancers and gig workers whose entire income is dependent on performing just one task, often for multiple clients or customers. And these kinds of roles are more vulnerable to being eliminated by AI. At the very least, AI could drive down wages rapidly in these areas. Now there is data in from several studies on this very phenomenon, and it looks pretty scary, as the Wall Street Journal reports.
Data from job posting board Upwork, as well as a study by academics at Harvard Business School, Washington University, and the University of Hong Kong that looked at jobs posted on Upwork, Fiverr, and other freelance job boards, found there have been steep falls in the number of some kinds of freelance work being offered on these platforms since ChatGPT debuted in November 2022. Freelance copywriting has been hit particularly hard, with the number of jobs being offered falling almost 20%. Other areas hit hard include translation services and customer support, as well as some web development roles. These are all tasks that today’s generative AI models can perform to some degree—although their output often still requires human supervision.
The Journal talked to a number of freelancers who reported fewer jobs—and that employers were also offering less money for the same work as before because they were asking AI to do a first draft, which the human only had to polish. (Some employers also assumed that the humans were using AI to assist them.) But it also talked to other freelancers who said they had actually been able to earn more because, with AI’s help, they could complete more tasks in a given time period, and thus take on more jobs. Still others said that they were benefitting when companies realized the quality of a lot of AI-generated content wasn’t good enough and they really did need specialized human knowledge, allowing these human experts to charge a premium for their services. Anyway, the story is well worth a read and a think.
FORTUNE ON AI
AI fears starting to fizzle among workers as half believe artificial intelligence will actually boost their salaries and increase job security—by Orianna Rosa Royle
The White House explains why it liked Microsoft’s investment in UAE AI champion G42: It got Huawei out of the way—by Lionel Lim
AI could kill creative jobs that ‘shouldn’t have been there in the first place,’ OpenAI’s CTO says—by Sydney Lake
Commentary: A.I. can radically lengthen your lifespan, says futurist Ray Kurzweil. Here’s how—by Ray Kurzweil
AI is touching your food—maybe most of it—by solving the food industry’s unique supply-chain challenges—by Stephanie Cain
Amazon’s money-bleeding Alexa division could face more cuts if a paid version of its AI-enabled voice assistant flops, BofA says—by Will Daniel
AI CALENDAR
June 25-27: 2024 IEEE Conference on Artificial Intelligence in Singapore
July 15-17: Fortune Brainstorm Tech in Park City, Utah (register here)
July 21-27: International Conference on Machine Learning (ICML), Vienna, Austria
July 30-31: Fortune Brainstorm AI Singapore (register here)
Aug. 12-14: Ai4 2024 in Las Vegas
BRAIN FOOD
People can’t tell if restaurant reviews are written by humans or AI. That’s a problem. Increasingly, text of all kinds can be generated by AI that is indistinguishable from human-written text. This threatens to make the issue of fake reviews—of everything from restaurants on Yelp and TripAdvisor to fake product reviews on Amazon—which are already contributing to a polluted “information ecosystem,” far worse.
A story in the New York Times reported on research conducted by Balazs Kovacs, professor at the Yale School of Management, that showed AI tools such as OpenAI’s GPT-4 and Anthropic’s Claude can easily produce restaurant reviews that most people can't tell apart from human-written ones. (Interestingly, in order to achieve this result, Kovacs first had to ask the AI to use colloquial expressions and even colloquial spellings—otherwise the prose sounded too polished.)
And the problem, as the Times notes, is that these reviews could easily overwhelm efforts by digital platforms that depend on user reviews for their utility to screen out fake reviews. It would be interesting if our latest technological leap into AI undoes the “wisdom of crowds” approaches that the internet itself made possible. We may end up back in a world where “word of mouth” recommendations from friends and family once again become our only way to assess something before trying it ourselves.
