Our mission to make business better is fueled by readers like you. To enjoy unlimited access to our journalism, subscribe today.
The visual effects artist who created three deepfake videos of Tom Cruise that went viral on TikTok last week said he’s in favor of laws that would regulate use of the technology.
“I strongly think that there should be laws that help with responsible use of A.I. and deepfakes in general,” Chris Ume told Fortune in one the first interviews he has granted to discuss how he created the deepfakes.
Deepfakes are convincing fake videos made with A.I. software. The videos, simple versions of which can be produced with freely available software, have been used to create fake revenge porn and to harass women, raising alarms among civil society groups and politicians. Security experts have also worried about the potential for such videos to be used in disinformation campaigns.
Ume said that there will always be people with bad intentions seeking to misuse technology but that his own purpose in creating deepfakes is entertainment, not disinformation or abuse.
While he often posts video “breakdowns” that showcase some aspects of how he creates his deepfakes, he said he never reveals the highly technical tricks he uses for the most difficult aspects of making the deepfakes believable, such as manipulating a deepfake character’s mouth. He also said that he wouldn’t work with a government hoping to use deepfakes for disinformation or propaganda.
This is Ume’s breakdown of his Tom Cruise TikTok videos:
As for the potential dangers of being fooled by deepfakes, Ume said that people must learn to become more skeptical and think more critically about the media they consume. He also noted that fears about deepfakes should not be overblown. “Normal consumers cannot do what I did,” he said. “You have to understand the quality of what I did and the work that goes into it.”
The three Tom Cruise videos had been posted to TikTok anonymously from an account called @deeptomcruise, and, for a time, trying to figure out who created them and how, became a social media parlor game. Earlier this week, Fortune reported that Ume was responsible for the deepfake portions of the videos after finding information online linking them to the Belgian-born visual effects artist. Ume confirmed the deepfakes were his work after Fortune contacted him.
Ume worked with Tom Cruise impersonator Miles Fisher to create three videos, which garnered more than 11 million views on TikTok. Experts in digital video forensics as well as social media fans of the genre of A.I.-created fake videos agreed they were some of the most well-executed and believable examples yet created. A number of commentators on Twitter and Reddit noted that the Cruise videos were able to fool “deepfake detection” software that some companies have created. But human video forensics analysts who examined the videos manually were still able to find telltale signs that the Cruise videos weren’t real.
The credibility factor
While security experts are worried the same methods used to create the Cruise videos could be used for political disinformation, so far this has not become a major trend. There is evidence of deepfake still images being used to generate profile pictures for fake accounts in social media disinformation campaigns. But so far the role of video has been limited to a few examples, most of which were not considered particularly credible.
One reason may be the extensive effort needed to create believable deepfakes. Ume said he uses free, open-source software called DeepFaceLab as the basis of his deepfakes. But he employs powerful graphics processing computer hardware to run this A.I. system, and, in this case, trained it on a database of 13,000 images of Tom Cruise that capture the actor from almost every conceivable angle. Ume said he then fine-tuned that Tom Cruise deepfake face generator on another, smaller data set containing between 5,000 to 6,000 additional Cruise images.
In total, training the Cruise deepfake generator took about two months, Ume said. Once his deepfake model is pretrained for this period, it can reliably generate new versions of Cruise’s head doing almost anything. At that point, it takes an additional two to three days to create the deepfakes specific to each of the short TikTok videos.
Ume then spent an additional 24 hours working on postproduction touch-ups for each video, painstakingly examining every frame. During this process, he worked on blending the deepfake face with Fisher’s body and removing any obvious anomalies or inconsistencies which deepfake software often produces. For instance, he said that sometimes the eyes in the deepfake don’t align correctly or the angle of a glance is wrong or the hair isn’t consistent, and he has to adjust those things manually with conventional computer video-editing software.
Ume prides himself on mouth animation, an especially tricky part of creating a convincing illusion. He said he was particularly gratified that Hany Farid, an expert in video forensics from the University of California at Berkeley, had told Fortune that he thought it was the actor’s real mouth used in the TikTok videos, not a deepfake one. Ume said Cruise’s entire face was a deepfake, except in the few frames where he would fade into Fisher’s real face.
Ume said that while he tries to remove obvious flaws and blatant visual anomalies when doing this postproduction work, he is not doing so in order to elude forensic analysts or deepfake detection software that might comb through a video frame by frame. “My intention is clear, and that is to make fun videos and make people smile and make people wonder,” he said. “I have no intention to fool anyone or fool the system.”
He also said that he isn’t surprised that deepfake detection software, which itself uses A.I., failed to flag the TikTok videos. He said that these systems have been trained on too many older examples of deepfakes, which were cruder than more recent high-quality examples.
Can you tell the difference?
Despite all his skill and high-tech wizardry, Ume said it would not have been possible to create such believable deepfakes if Fisher hadn’t borne such a close resemblance to Cruise and if Fisher hadn’t been such a capable impressionist, able to convincingly imitate Cruise’s voice, cadence, and mannerisms. This allowed Ume to play around with carefully blending Fisher’s face with the Cruise deepfake and fading in and out between the two. He said there are moments in some of the videos, particularly one in which Cruise appears to be practicing his golf swing, where the viewer is simply seeing Fisher’s face, without any deepfake overlay, but that the viewer can’t easily tell the difference.
It also helps that there is so much publicly available imagery of Cruise. Ume said that he could not create such a realistic deepfake of someone for whom there was far less available data, because the software wouldn’t be able to realistically depict his or her face at enough angles and with different expressions.
Deepfakes use an A.I. technique called a GAN, or generative adversarial network. It involves yoking two neural networks—a kind of machine learning loosely modeled on the human brain—together and training them in tandem. One network learns, from a database of real images of someone’s face, how to generate new, realistic-looking fake images. The second network is trained to try to identify real images of that particular person from a database of faces that includes some images of the deepfake target, and lots of images of other people. The first network has to learn to generate fake images that are good enough to fool the second network.
Ume, who trained in traditional digital effects at a Belgian university, said he first encountered deepfakes in 2018, about five years after graduating. He was immediately drawn to them. “There was this new [story] on TV talking about how the technique was being misused. But I saw that and thought that this is so cool,” he recalled. “There are so many things you can use this for in creative ways, even creative ways we haven’t tried before.”
Ume is a member of Deep Voodoo studio, a cadre of deepfake experts assembled by Trey Parker and Matt Stone, the cocreators of South Park, and English comedian and filmmaker Peter Serafinowicz to produce their satirical YouTube series Sassy Justice, which uses a combination of real actors and deepfaked celebrity heads. He was recruited after receiving attention for a viral video he created in 2019 in which he had a deepfake of the actor Kit Harington, who plays Jon Snow on Game of Thrones, apologize for the hit series’ widely panned final season.
The deepfake master said he wound up working with Fisher on the Cruise videos after he saw a video Fisher created in August 2019 in which he portrays the famous actor announcing a mock presidential run. Ume took the video and redid it with a deepfake Cruise face superimposed on Fisher’s and then posted a breakdown of his deepfake version side by side with the original on his own YouTube channel. He said he tried to contact Fisher through social media to get his permission but never heard back from him.
A year later, out of the blue, Fisher contacted Ume and asked him to work on a new Cruise parody video in which Fisher would have Cruise conceding his presidential bid, his face morphing into a Cruise deepfake in the sketch’s final moments. That video was posted online, with full credits to both Fisher and Ume, in January. At the same time, Fisher asked Ume to create some additional deepfake videos to promote his podcast. They filmed four altogether, three of which became the viral videos posted on TikTok.
Ume said he thinks deepfakes will likely transform how Hollywood makes movies. “It gives you so much more creative possibilities,” he said. “If you have an actor who would be amazing for a role but doesn’t physically fit the role, you can transform the type of character, and they can now play that role. There are so many things we can do with it.” He said his own personal ambition is to one day make a movie with Lord of the Rings director Peter Jackson, who is known for his computer-generated special effects.
