Taylor Swift deepfake porn points to a fundamental problem: AI can make it, but can’t police it

Wall Street investors and almost anyone who’s tracking the progress of the generative AI boom is waiting to see what sort of quarterly earnings tech giants Microsoft and Alphabet post after the market close today. Many are hoping to see both companies put big topline growth figures on the board due to sales of AI-enhanced products and cloud services. Analysts think Microsoft could see 15% revenue growth and an almost 19% earnings jump, largely due to more companies using its AI cloud services and its new AI-enhanced software tools. That would be a significantly large increase for such a giant company. On the other hand, if both companies disappoint, it may feed into concerns the AI boom is overhyped.

While we’re waiting for that news, let’s talk about some other stuff. Like porn. And Taylor Swift. Deepfake pornographic images of the music star went viral on social media platform X and on various Telegram channels this past week, underscoring the huge problem nonconsensual deepfake porn poses not just to Swift, but to women everywhere. Some are hopeful Swift will use her considerable cultural influence to create a groundswell of support for regulation that might actually do something to stem the tide of these sorts of deepfakes, which are often used to harass non-celebrities. And, in fact, several Congressional representatives introduced bills aimed at combating deepfake porn in response to the Swift deepfakes, and White House spokesperson Karine Jean-Pierre said legislation on the misuse of social media might be needed.

The question is exactly what form those laws should take. In the U.K., the new Online Safety Bill puts responsibility on the people who create the images and post them online, making the sharing of nonconsensual pornography a crime. But it is unclear how easy the law will be to enforce or how much attention police and prosecutors will devote to pursuing such cases. The creators of these images usually take steps to hide their identity, making such investigations technically difficult. The law also stops short of holding social media companies that allow these kinds of deepfakes to go viral criminally liable. However, it does require them to show that they have systems to try to prevent the spread of nonconsensual porn and to remove the content quickly if it does slip through their filters.

This is the kind of regulation that even some big tech CEOs have advocated in response to the problem of deepfakes and disinformation of all kinds. Stop it at the point of distribution, not the moment of creation. That’s what Microsoft CEO Satya Nadella said in recent comments at London’s Chatham House and in Davos. To paraphrase Nadella’s argument: Going after the people making AI models because they happen to be able to make deepfake porn is like suing Smith Corona because a bank robber used one of its typewriters to write a stickup note.

Then again, he would say that. Microsoft doesn’t have a major social network to police. But it does make and sell AI software. And as it turns out, there’s good evidence that it was Microsoft’s Designer software, which includes the ability to use natural language prompts to create images, that was used to create the Swift deepfakes. After tech publication 404 Media showed how easy it was to get around Microsoft’s guardrails to create Swift deepfakes, Microsoft strengthened some of those prompt restrictions.

What is needed is a multi-layered approach that addresses all three levels of the problem: laws that make it a criminal offense to create and distribute nonconsensual porn and deepfakes; laws that require AI model makers to have far more robust guardrails than they do currently; and, most importantly, laws that require social media companies to better filter out such imagery and prevent it from going viral.

The ease with which Designer’s guardrails can be overcome and the problem social media giants have in filtering out pornographic content stem from the same fundamental issue: Despite all their seeming sophistication and abilities to pass the bar exam or U.S. medical licensing exams, AI systems still lack anything approaching human-level understanding. Pornography is famously hard to define, even for humans. As Supreme Court Justice Potter Stewart famously quipped, he couldn’t define it, “but I know it when I see it.”

In theory, this is exactly the sort of problem at which modern AI, based on neural networks, should excel. One reason neural network-based deep learning caught on in the first place is that such software could classify images, such as telling photos of cats apart from ones of dogs, not based on some elaborate rules and definitions, but by developing an impossible-to-explain, almost intuitive sense of when an image depicted a cat or a dog.

But it turns out pornography is a much more complex concept to grasp for AI than identifying a cat or dog. Some nudity is innocent. Some is not. And our deep learning classifiers have struggled to understand enough about semantic composition—the parts of an image that give it a particular meaning—and context to make those calls successfully. That’s why so many social media platforms end up blocking the distribution of innocent baby snaps or photos of classical sculptures that depict nude figures—the AI software powering its filters can’t tell the difference between these innocent images and porn. Laws such as the U.K.’s Online Safety Act wind up incentivizing companies to err on the side of blocking innocent images since it keeps them from getting fined and drawing lawmakers’ ire. But it also makes these platforms less useful.

The same goes for our image generation AI, which is also based on deep learning. You can’t simply create guardrails for these systems by telling them behind the scenes, “Don’t create porn.” Instead, you have to ban user prompts such as “Taylor Swift nude.” But, as it turns out, the same system will still create essentially the same image when prompted with “Taylor ‘singer’ Swift” and then, as 404 Media reported, “rather than describing sexual acts explicitly, describe objects, colors, and compositions that clearly look like sexual acts and produce sexual images without using sexual terms.” Again, this is because the image generator doesn’t have any understanding of what porn is. And as companies try to strengthen these guardrails, they render their own products less useful for legitimate use cases.

This is one of those AI problems that it may take an entirely new AI architecture to solve. Yann LeCun, Meta’s chief AI scientist, has been advocating for a new deep learning method for image classifiers called a Joint Embedding Predictive Architecture (or JEPA) that tries to create AI models with a much more robust conceptual and compositional understanding of a scene. It is possible that an image classifier based on JEPA might be a better Taylor Swift deepfake porn detector than our current models.

We’ll have to wait to hear from Yann whether this works for Taylor Swift deepfakes. In the meantime, expect deepfake porn to continue to be a scourge of social media.

There’s lots more to AI news, so read on. But, before you do: Fortune is always trying to make Eye on AI more valuable to our readers. If you could take a couple of minutes to give us your honest feedback by answering a few questions about your experience, I’d appreciate it. It shouldn’t take you more than five minutes. You can find the link below. Thanks!

Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn

Correction: Last Tuesday’s edition Eye on AI misstated a statistic Getty CEO Craig Peters used to illustrate the growth of AI-generated imagery. He said more AI-created images have been produced in the past 12 months than lens-based photographs, not that the number of AI-generated images produced in that period already exceeded the number of photographs produced throughout history.

AI IN THE NEWS

Elon Musk looks to raise $6 billion for his xAI startup. Musk has hit up family offices in Hong Kong, according to the Financial Times, in his quest to land the mega funding round that would reportedly value his AI startup at $20 billion. Rival AI company Anthropic is reportedly valued close to that amount. Many speculated that the deal size implied xAI might be seeking to build its own AI data centers, rather than renting cloud computing time from one of the existing hyperscale cloud providers, most of which are allied with competing AI companies. The financial newspaper also said Musk had spoken to Middle Eastern sovereign wealth funds and investors in South Korea and Japan about the deal. The billionaire entrepreneur, however, in a comment posted to X, the social media platform he controls, denied he was seeking any outside funding for xAI or had conversations with investors.

Musk’s Neuralink implants first brain chip in a human. Musk posted the news of the company’s first human trial on social media platform X, according to The Information. Musk said the patient is “recovering well” and that the company’s Link brain-computer interface was detecting the patient’s neural activity. The trial is meant to test the safety of both the link and the robotic surgery method Neuralink is pioneering to implant the device. The trial is open to patients who are quadriplegics as a result of a cervical spinal cord injury or have amyotrophic lateral sclerosis. For more on Neuralink, you can check out this feature I cowrote on the company in 2022.

U.S. wants cloud providers to flag Chinese AI activity in their data centers. The major cloud services would need to tell the government of any foreign entities training AI models on their platforms or in the data centers they control, handing over IP addresses and customer names as well as reporting any suspicious activity taking place on their servers, according to a Biden Administration proposal released earlier this week, Bloomberg reports. The proposed rules come amid an increasingly intense technological cold war between the U.S. and China over AI development. They seek to close down a path by which Chinese companies, or entities affiliated with them, could have continued to access the most advanced AI computer chips after the Biden Administration cut off the export of these chips to China. One of the cloud hyperscalers, Microsoft, said it welcomed “know your customer” requirements. A Chinese government spokesperson said China wished the U.S. would cooperate with it on AI “rather than decoupling, breaking chains and building fences.”

U.K. cybersecurity unit warns of risks of AI-augmented hacking. The National Cyber Security Center (NCSC), which is part of the British signals intelligence agency GCHQ, says in a new report that AI will “almost certainly” increase the volume and impact of cyberattacks over the next two years. It says this threat comes mostly from enhancing existing attack methods—such as making phishing and other kinds of social engineering more effective—as well as from using LLMs to analyze exfiltrated data faster than before, allowing hackers to refine attacks on the fly. It says that new attack techniques or malware are most likely to come from nation-state attackers with access to large amounts of training data, computing resources, and money, and that these are most likely to appear only after 2025.

India has its first generative AI unicorn. Krutrim, an AI startup founded by serial entrepreneur Bhavish Aggarwal, who previously created ride-sharing company Ola, has become the first Indian AI startup to achieve unicorn status—valued at over $1 billion after a $50 million venture capital round, Bloomberg reports. The company, whose name means “artificial” in Sanskrit, recently introduced its first large language model, which can generate text in 10 languages, including English and Hindi, as well as Hinglish, a mix of Hindi and English that is colloquially spoken by many Indians.

And India is preparing rules to require social media companies to police deepfakes. Rajeev Chandrasekhar, India’s IT minister, warned social media companies that the government would hold them accountable for deepfakes and misinformation shared on their platforms, the Financial Times said. The country made it illegal in 2021 for social media platforms to host misinformation and Chandrasekhar said his office wanted to remind the companies of this ahead of the country’s national elections, scheduled for April and May. He also said India was trying to forge a middle path on tech regulation between the laissez-faire approach of the U.S. and the strict prohibitions favored by Europe. But some civil society groups have accused the government of Prime Minister Narendra Modi of wielding the country’s social media laws to censor information critical of the government or supportive of opposition political parties. Chandrasekhar has denied these accusations.

EYE ON AI RESEARCH

Another LLM security flaw to lose sleep over. Security researchers have discovered a vulnerability in some kinds of graphics processing units (GPUs), the chips used most frequently for AI, that would allow an attacker to steal data from someone’s conversation with an AI chatbot. The research, published by security firm Trail of Bits, affects some GPUs from Apple, Qualcomm, AMD, and the company Imagination. It does not seem to affect Nvidia, which makes the most popular GPUs for AI applications, or those from Arm.

To implement the attack, the attacker has to be able to run an application that uses the same GPU as the target chatbot or other AI software. It suggests a trojan horse masquerading as a legitimate app could do the trick. The attack exploits the fact that GPUs store some data to local memory to speed up their processing, and this “leftover” data can then be read by another application that is also using the GPU. Because of this Trail of Bits called the attack “LeftoverLocals.” The attack, according to Trail of Bits, is difficult to detect and likely difficult to fix—although Trail of Bits suggests that all applications that use the GPU’s local memory should include code that wipes that memory clean immediately after use.

It’s yet another example of how our increasing use of LLMs is spawning a new class of security vulnerabilities that we are just beginning to understand and get to grips with.

FORTUNE ON AI

Amazon walking away from its $1.7 billion iRobot deal leaves the Roomba maker without its founding CEO and staring down a 31% staff cut —by Marco Quiroz-Gutierrez

Let’s talk about why Elon Musk needs $6 billion for xAI —by Allie Garfinkle

The chip industry’s dirty little secret: It’s very dirty —by Michal Lev-Ram

The FTC’s inquiry into Microsoft, Amazon, and Alphabet’s AI alliances is the opening shot in a showdown that could reshape tech —Jessica Mathews

Skill shortages are one of industry’s biggest problems. AI is the solution —by Vimal Kapur (Commentary)

AI CALENDAR

Jan. 30: Microsoft and Alphabet report quarterly earnings

Feb. 1: Meta and Amazon report earnings

Feb. 21: Nvidia reports earnings 

March 18 - 21: Nvidia GTC AI conference in San Jose, Calif.

April 15-16: Fortune Brainstorm AI London

June 25 - 27: 2024 IEEE Conference on Artificial Intelligence in Singapore

PROMPT SCHOOL

Go with the flow. This time this section should maybe be called “anti-prompt school.” Last week, I spoke to Itamar Friedman, the cofounder and CEO of Tel Aviv-based AI coding assistant CodiumAI about why companies should get away from thinking about “prompt engineering”—designing a list of instructions to tell a large language model what to do—and instead think about “flow engineering.” What’s that? It’s about designing a process where an LLM is just one component in a flow of steps that can involve other LLMs, other kinds of AI models, non-AI software, and even human feedback.

This is how Codium created a software model called AlphaCodium that outperformed the best previous AI code generation system, Google DeepMind’s AlphaCode, which was released in December 2022, on the same CodeContests benchmark that the well-known deep learning lab used to test AlphaCode. The CodeContests dataset includes tough software programming problems that often stump human competitors in coding competitions run by Codeforces. AlphaCodium was also able to take OpenAI’s GPT-4, which when simply prompted to answer the CodeContests problems can only generate accurate solutions to 19% of them, and brings that number to 44% by working GPT-4 into the AlphaCodium flow Codium designed.

Whereas AlphaCode generates hundreds of thousands of possible solutions and then narrows them down to a best guess, AlphaCodium works by creating a flow where one fine-tuned AI model generates code and another model learns to debug and verify that code. The two work together in a loop to solve the coding challenges.

Friedman says this kind of flow produces better results and uses less computing power than simply asking a huge general-purpose LLM to tackle an entire problem. Instead, the problem is broken down into steps, with specialized, small AI models handling just that particular step and then handing off to the next. If companies start thinking about designing flows like this, instead of using a giant LLM for everything, they can arrive at more accurate outputs and save money, Friedman tells me. Yes, it does involve a bit more developer skill to chain the different models together—but not that much more. And the effort is probably worth it. I predict we’re going to be hearing a lot more about flow engineering over the coming months.