from the seeing-is-believing...-but-for-how-long? dept.
Remedy, the developer behind the likes of Alan Wake and Quantum Break, has teamed up with GPU-maker Nvidia to streamline one of the more costly parts of modern games development: motion capture and animation. As showcased at Siggraph, by using a deep learning neural network—run on Nvidia's costly eight-GPU DGX-1 server, naturally—Remedy was able to feed in videos of actors performing lines, from which the network generated surprisingly sophisticated 3D facial animation. This, according Remedy and Nvidia, removes the hours of "labour-intensive data conversion and touch-ups" that are typically associated with traditional motion capture animation.
Aside from cost, facial animation, even when motion captured, rarely reaches the same level of fidelity as other animation. That odd, lifeless look seen in even the biggest of blockbuster games often came down to the limits of facial animation. Nvidia and Remedy believe its neural network solution is capable of producing results as good, if not better than what's produced by traditional techniques. It's even possible to skip the video altogether and feed the neural network a mere audio clip, from which it's able to produce an animation based on prior results.
The neural network is first fed a "high-end production facial capture pipeline based on multi-view stereo tracking and artist-enhanced animations," which essentially means feeding it information on prior animations Remedy has created. The network is said to require only five to 10 minutes of footage before it's able to produce animations based on simple monocular video capture of actors. Compared to results from state-of-the-art monocular and real-time facial capture techniques, the fully automated neural network produces eerily good results, with far less input required from animators.
With the network trained, it's possible to feed in an audio performance, which is then mapped to a 3D model to generate an animation. The process involves "a compact, latent code" that formulates the variations in facial expressions that can't be deduced from the audio recording. That code could potentially be user controlled, which would allow for the resulting animation to be tweaked.
https://arstechnica.com/gaming/2017/08/nvidia-remedy-neural-network-facial-animation/
(Score: 0) by Anonymous Coward on Friday August 04 2017, @06:01AM
This is combined with that previous article about Neural Network speech synthesis to help improve plea bargaining and conviction rates to 100 percent with 'authentic' confession tapes?
We've already created a technogical hell, we just haven't realized the system controlling it is too big for anything short of a billion people waking up to fix. And with the narratives they will soon be able to craft, 1984 will soon be more truth than fiction.