| Title | My Struggle With Deepfakes | |
| Date | Tuesday May 29 2018, @10:29PM | |
| Author | takyon | |
| Topic | ||
| from the crisp-fakes dept. | ||
There has been some controversy over Deepfakes, a process of substituting faces in video. Almost immediately, it was used for pornography. While celebrities were generally unamused, porn stars were alarmed by the further commodification of their rôle. The algorithm is widely available and several web sites removed objectionable examples. You know something is controversial when porn sites remove it. Reddit was central for Deepfakes/FakeApp tech support and took drastic action to remove discussion after it started to become synonymous with fictitious revenge porn and other variants of anti-social practices.
I found a good description of the deepfakes algorithm. It runs via a standard neural network library but requires considerable processing power on specific GPUs. I will describe the video input (with face to be removed) as the source and the face to be replaced as the target. The neural network is trained with the target face only. The source is distorted and the neural network is trained to approximate reference images of the target. When the neural network is given the source, it has been trained to "undistort" the source to target.
[Continues...]
If there are multiple faces in a frame of video, face recognition restricts input to the most likely face. Indeed, for maximum efficiency, this technique is used to crop source video in all cases. The trick that makes the process feasible is that the neural network is only trained with the target face. Furthermore, given the use of libraries, the unique code to achieve this objective is shockingly small.
A friend attempted to mix DeepFakes with the Internet meme of Downfall parodies. There is an infamous scene in the film Downfall (not to be confused with the film Falling Down) where Adolf Hitler rants prior to defeat. Unfaithful subtitles of the German dialog have been used to parody everything from corporate sales targets to sportsball management to the ongoing medical abuse of transsexual patients. Until now, only the words in the subtitles changed. The audio and video was otherwise unchanged. My friend hoped that it would be possible to insert the likeness of people being parodied.
Unfortunately, it doesn't work with the current algorithm. The number of faces is not a problem. The clipping and occlusion prevents the neural network from working effectively. It should be possible with an extension of the current algorithm but it is currently impractical.
A further development, found by the same friend, is the automatic conversion of a one sentence description into a very short video. The example system uses Flintstones cartoons. An example sentence would be "Fred dancing in the kitchen" and a rough but valid video is created which matches the description. Potentially, it would be possible to automatically convert a novel into a 100 minute film with no human intervention. Given that novels are frequently converted into films, there is a large amount of example data which may be used as reference. I know this would only be moderately easier than making a holodeck but experts may not be aware of the progress towards either goal.
takyon: An algorithm can also be used to manipulate facial movements to match video or audio input (see this example of Jordan Peele controlling Barack Obama's face). DARPA is holding an event that will task experts with making and catching "deepfakes".
Researchers have also created short "movies" (64x64, 32-frame animated GIFs) from text descriptions. It may be possible to synthesize scenes for a full length movie in the future without needing strong AI. After all, procedural generation could be used to create and populate a virtual city (like the one in Big Hero 6), and then it's a matter of writing some kind of coherent narrative and "shooting" it. A "director neural network" could be trained to mimic the cinematography techniques of films created by humans, and then apply the results to the virtual environment.
| Links |
printed from SoylentNews, My Struggle With Deepfakes on 2024-04-18 11:17:13