Epic Games' Tim Sweeney on creating believable digital humans
Epic Games stunned everyone a couple of years ago with the realistic digital human character Senua, from the video game Hellblade. And today, the maker of the Unreal Engine game tools showed another astounding demo, dubbed Siren, with even more realistic graphics.
CEO Tim Sweeney said technologies for creating digital humans — from partners such as Cubic Motion and 3Lateral — are racing ahead to the point where we won't be able to tell the real from the artificial in video games and other real-time content.
[...] [Kim Libreri:] The other big thing for us, you may have seen the Microsoft announcements about their new raytracing capabilities in DirectX, DXR. We've partnered with Nvidia, who have the new RTX raytracing system, and we thought about how to show the world what a game could look like in the future once raytracing is added to the core capabilities of a PC, or maybe even a console one day. We teamed up with Nvidia and our friends at LucasFilm, the ILM X-Lab, to make a short film that demonstrates the core capabilities of raytracing in Unreal Engine. It's an experimental piece, but it shows the kind of features we'll add to the engine over the next year or so.
We've added support for what we call textured area lights, which is the same way we would light movies. You can see multiple reflections. You can see on the character, when she's carrying her gun, the reflection of the back of the gun in her chest plate. It's running on an Nvidia DGX-1, which is a four-GPU graphics computer they make. But as you know, hardware gets better every year. Hopefully one day there's a machine that can do this for gamers as well as high-end professionals. It's beginning to blur the line between what a movie looks like and what a game can look like. We think there's an exciting time ahead.
One thing we've been interested in over the years is digital humans. Two years ago we showed Senua, the Hellblade character. To this day, that's pretty much state of the art. But we wanted to see if we could get closer to crossing the uncanny valley. She was great, but you could see that the facial animation wasn't quite there. The details in the skin and the hair—it was still a fair way from crossing the uncanny valley.
Video is available on YouTube: Siren, alone (42s) and Siren Behind The Scenes (52s), and Creating Believable Characters in Unreal Engine (56m31s).
Related: Microsoft Announces Directx 12 Raytracing API
Related Stories
At GDC, Microsoft announced a new feature for DirectX 12: DirectX Raytracing (DXR). The new API offers hardware-accelerated raytracing to DirectX applications, ushering in a new era of games with more realistic lighting, shadows, and materials. One day, this technology could enable the kinds of photorealistic imagery that we've become accustomed to in Hollywood blockbusters.
[...] Because of the performance demands, Microsoft expects that DXR will be used, at least for the time being, to fill in some of the things that raytracing does very well and that rasterization doesn't: things like reflections and shadows. DXR should make these things look more realistic. We might also see simple, stylized games using raytracing exclusively.
The company says that it has been working on DXR for close to a year, and Nvidia in particular has plenty to say about the matter. Nvidia has its own raytracing engine designed for its Volta architecture (though currently, the only video card shipping with Volta is the Titan V, so the application of this is likely limited). When run on a Volta system, DXR applications will automatically use that engine.
In conjunction with Microsoft’s new DirectX Raytracing (DXR) API announcement, today NVIDIA is unveiling their RTX technology, providing ray tracing acceleration for Volta and later GPUs. Intended to enable real-time ray tracing for games and other applications, RTX is essentially NVIDIA's DXR backend implementation. For this NVIDIA is utilizing a mix of software and hardware – including new microarchitectural features – though the company is not disclosing further details.
Royale-lly Screwed: Epic Sues YouTubers Promoting Fortnite Cheats
Part of Fortnite's appeal is that it offers a level playing field. People can't unlock new weapons, start a match with equipment, or augment their abilities by grinding for in-game upgrades. Everyone drops out of the Battle Bus with the same tools, a glider and a pickaxe so their skill will determine whether they catch the 'dub or take an L. So it may not come as a surprise that Epic Games is suing two YouTubers for upsetting that balance with cheats.
TorrentFreak has reported that Epic's lawsuit targets Brandon "Golden Modz" Lucas, a cheat distributor and content creator whose YouTube channel has 1.7 million followers, and Colton "Exentric" Contor, who has over 7,000 followers. The cheat in question combined an aimbot with ESP features that offered information that players would otherwise have no way of knowing. It reportedly cost $55 (30 days) or $300 (unlimited) from the Golden Godz website.
[...] The suit appeared to have a quick impact. Golden Modz's last video was published on October 12, and the Golden Godz website currently says that "No packages exist at this time," even though a dialog box claims that "Payment systems are back up and new packages have been added!" The site claims to offer various "services" for several Call of Duty games and Grand Theft Auto: Online; all of them appear to have been pulled.
Also at Polygon.
Previously: Epic Games Sues 14-Year-Old after He Files a DMCA Counterclaim for a How-to-Cheat Video
U.S. Federal Judge Blocks Man From Selling GTA V Cheating Software
Related: Game About Net Neutrality Receives Grant from Epic Games
Epic Games (Developer of the Unreal Engine) Shows Off "Siren" Demo
Sony Faces Growing 'Fortnite' Backlash At E3
Fortnite's Android Version Bypasses Google Play to Avoid 30% "Store Tax"
Epic's first Fortnite Installer allowed hackers to download and install anything on your Android phone
(Score: 4, Funny) by Snospar on Saturday March 24 2018, @06:24PM (16 children)
I know it's the uncanny valley but seriously, what is up with the dead eyes and the teeth? Look at the teeth! Does she gnaw on bones for sustenance?
I need a beer after that.
Huge thanks to all the Soylent volunteers without whom this community (and this post) would not be possible.
(Score: 1, Troll) by Ethanol-fueled on Saturday March 24 2018, @06:30PM (2 children)
All those pink-haired dykes from Bioware had to go somewhere after their retardation crashed the company.
(Score: 3, Funny) by Snospar on Saturday March 24 2018, @07:26PM
So Pink(-haired) Soylent is Bioware people. Yuck, I'll pass thanks.
Huge thanks to all the Soylent volunteers without whom this community (and this post) would not be possible.
(Score: 1, Funny) by Anonymous Coward on Saturday March 24 2018, @09:40PM
What about the blue-haired ones? I figured we were supposed to be afraid mostly of the blue-haired ones.
(Score: 3, Insightful) by JoeMerchant on Saturday March 24 2018, @07:59PM (2 children)
Didn't you hear the accent? Classic UK teeth.
🌻🌻 [google.com]
(Score: 3, Informative) by Snospar on Saturday March 24 2018, @08:35PM
I think you might be on to something... that's an English accent but those look like Scottish teeth to me. No wait, I was wrong, no fillings and no gaps, not Scottish after all.
(Disclaimer: The Scots are known for their sweet tooth, this isn't meant to cause offence. If it does offend you - grow up.)
Huge thanks to all the Soylent volunteers without whom this community (and this post) would not be possible.
(Score: 3, Insightful) by driverless on Sunday March 25 2018, @02:41AM
I don't get it. They took a real human actress and mapped her 1:1 to a somewhat creepy-looking CGI-ish one. It's like Gus Van Sant's Psycho, but done worse.
(Score: 0) by Anonymous Coward on Saturday March 24 2018, @09:30PM (6 children)
They spend so much time trying to create something so hideous. Reality is ugly and boring and real people are an abomination. Anime-style stuff is better.
(Score: 3, Insightful) by takyon on Saturday March 24 2018, @09:43PM (4 children)
You can still use a big, hot GPU for that. Just render the characters kawaii and use the raytracing and other stuff for the background details.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 1) by Ethanol-fueled on Sunday March 25 2018, @12:50AM (3 children)
And just how do people get GPU's nowadays? The same way Venezuelans get Filet Mignon?
(Score: 2) by takyon on Sunday March 25 2018, @01:09AM (2 children)
They got the GTX 970 or whatever back when it was decently priced and in stock. Or invent time travel.
Judging from these 1000-series charts, it seems that shit only hit the fan (and stayed there) starting around December:
https://camelcamelcamel.com/search?sq=GTX+970 [camelcamelcamel.com]
So if you had jumped on some Black Friday GPU deal, you could have missed the crypto madness.
Oh, here's a deal I guess: https://slickdeals.net/f/11375679 [slickdeals.net]
If a GTX 1050 is not fast enough for you, then just stick with what you have until the new cryptocash die$ or moves to ASICs.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 0) by Anonymous Coward on Sunday March 25 2018, @06:41PM
"then just stick with what you have until the new cryptocash die$ or moves to ASICs."
or until these stupid sacks of shit learn how to ship GPUs to card manufacturers.
(Score: 2) by Freeman on Monday March 26 2018, @04:36PM
I've been on the lookout for a decently priced Vega GPU from the moment they were released. Sad fact, normal people couldn't get one, because they were constantly out of stock. When they were in stock it was with a stupidly priced bundle. Now, they're in stock with stupidly high prices. Sure hope my RX480 doesn't decide to give up the ghost in the meantime. Though, I should hopefully be able to get it RMA'd, if that were to happen any time soon.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 2) by fyngyrz on Sunday March 25 2018, @12:26AM
You should go back downstairs now. You know you're only allowed up for meals and laundry exchange.
(Score: 3, Interesting) by RamiK on Saturday March 24 2018, @09:53PM (1 child)
I think it's the lips during plosive consonants not perking right depending on proceeding vowels/consonants and the cheeks during open vowels.
Nothing really wrong with the eyes as far as I can tell... But my eyesight isn't too good so I just might be missing some details.
I think the sample is a bit biased though since it's a well reversed speech with exaggerated facial movements so in slower scenes we might feel we're missing out on some subtleties like micro-expressions. But that's just speculation.
Overall, pretty good.
compiling...
(Score: 2) by VanessaE on Sunday March 25 2018, @06:35AM
The first thing I noticed was the plastic-looking skin.
(Score: 2) by frojack on Sunday March 25 2018, @05:14AM
Never mind the teeth, why can't they get lip movement to have some resemblance to the simulated speech?
The skin shines, the eyes are utterly fake and the lips look like she's speaking g Russian..
No, you are mistaken. I've always had this sig.
(Score: 1) by Ethanol-fueled on Saturday March 24 2018, @06:26PM (4 children)
More advanced technology is always good news, but how are cryptocurrencies going to affect the availability and pricing of graphics cards? I went to Fry's because I wanted a dual-monitor setup and I wanted it now, but there were only two low-end graphics cards on the shelf. No, not two sets of models, literally two fucking physical cards.
Before that I had thought the lack of available cards was just hyperbole.
(Score: 3, Informative) by takyon on Saturday March 24 2018, @06:50PM (3 children)
If the cryptocurrencies that are best on GPUs (rather than ASICs) don't go bust, resulting in a flood of used GPUs on the market, maybe the flood will happen once new GPUs are released.
NVIDIA and especially AMD know that cryptocurrency made them a lot of money in recent quarters. Maybe they will optimize for lower power consumption on their upcoming generation of GPUs, making something miners want to buy.
http://www.tomshardware.com/news/nvidia-gtx-2080-predictions,36720.html [tomshardware.com]
Tom's mentions a possibly July launch for GTX 2000-series, and prices could be obscene, like $1500 for the GTX 2080.
https://wccftech.com/nvidia-gpp-ignites-uproar-calls-for-boycott-among-pc-gamers/ [wccftech.com]
A boycott of Nvidia over GPP? That would be nice, but it might have no effect if PC gamers are already priced out.
Navi could be launched in late 2018, but AMD seems to be screwing up as usual:
https://segmentnext.com/2018/02/07/amd-navi-gpu-roadmap/ [segmentnext.com]
https://www.pcgamesn.com/amd-navi-gcn-next-gen-architecture [pcgamesn.com]
In short, it's not looking good. Maybe wait 2-3 years?
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 0) by Anonymous Coward on Saturday March 24 2018, @07:00PM (1 child)
I'm not keen on using a card that's been pumped full of someone else's electrons.
(Score: 3, Funny) by takyon on Saturday March 24 2018, @07:02PM
How do you feel about backdoors?
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 2) by MichaelDavidCrawford on Sunday March 25 2018, @06:29AM
Some currencies are "ASIC Resistant". Among the ways that they are deliberately designed such that ASICs wouldn't work for them is that mining them requires more memory than ASICs could feasibly possess.
Someone once posted to Bitmain's message board that they were bitmain's exclusive distributor for bitmain's new Etherium ASIC rig. A bitmain employee replied to say that they have no resellers, and that there is no such thing as an Etherium ASIC rig.
Specifically for Etherium, it requires so much memory that a midrange gaming GPU is insufficient. I expect it is quite likely that GPUs that were highly optimized for mining would not be of much use for gaming or video.
Monero is another ASIC-resistant crypto. It doesn't require as much memory as Etherium. I'm not completely clear on why it's not compatible with ASICs.
A big advantage of GPU mining is that it presents the individual a far-lower cost of entry: you can mine with just one GPU. It won't yield many coins but then it doesn't cost so much.
"GPU Mining Rigs" are simple card cages with six many-lane PCIexpress slots, a CPU motherboard and a power supply.
I puzzled over Monero mining not long ago, and decided that I would stick with ASIC Litecoin mining because one $2,000 LTC rig had a greater US-dollar yield than $2,000 worth of GPUs would yield from Monero.
Put another way, I have the wherewithal such that I don't really need a lower cost of entry. It's good to have a job.
Yes I Have No Bananas. [gofundme.com]
(Score: 0) by Anonymous Coward on Saturday March 24 2018, @06:39PM (2 children)
there's something missing and she looks a bit plastic.
(Score: 3, Insightful) by takyon on Saturday March 24 2018, @06:53PM
the valley 🅳🅴🅴🅿
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 2) by JoeMerchant on Saturday March 24 2018, @08:02PM
I thought the animation of her arm motions were disappointingly artificial looking, and that should be the easier part to get right - especially if they're doing motion capture from an actress.
🌻🌻 [google.com]
(Score: 2, Interesting) by koick on Saturday March 24 2018, @06:42PM (5 children)
(Score: 4, Interesting) by takyon on Saturday March 24 2018, @06:52PM
https://arstechnica.com/gaming/2017/05/how-neural-networks-are-making-for-more-lifelike-video-game-animation/ [arstechnica.com]
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 2) by JoeMerchant on Saturday March 24 2018, @08:08PM
Well, some people look good, and some people can act. People who both look good and can act are fabulously expensive to hire, motion capture ultimately brings the cost of production down as compared to having a bidding war with other production studios for the top talent.
Plus, this saves Jim Kirk's girlfriends from having to wear green body makeup... that's a big plus for them right there.
🌻🌻 [google.com]
(Score: 2) by leftover on Saturday March 24 2018, @08:30PM (2 children)
Have to agree. The actress's face is perfectly appealing as-is. Just paint the makeup on her and let the computers handle the background rather than diminishing her performance.
Bent, folded, spindled, and mutilated.
(Score: 2) by takyon on Saturday March 24 2018, @09:24PM (1 child)
That's acceptable for a cutscene, not a game engine.
I don't get this criticism.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 3, Interesting) by leftover on Saturday March 24 2018, @11:25PM
Nothing earthshaking. Just musing on the progress of technical capability. Have you ever seen the filmstrip of Pearl Bailey doing her role in The Fox and the Hound side by side with the resulting animation? All wetware but the animators captured her really very well despite the complete change in species. That is what I was thinking about in comparison to the clips of Siren beside those of the live actress. As a technical achievement it is marvelous but as entertainment it is ... uncomfortable? It looked to me like the facial model had enough control but the motion capture was not collecting data at the same level of detail. People, even old relics like me, are sensitive to incredibly nuanced detail in facial expressions. Maybe the change in species actually helps the illusion by decoupling human face rules from not-human face rules.
Bent, folded, spindled, and mutilated.
(Score: 2, Touché) by Captival on Saturday March 24 2018, @07:23PM (5 children)
The uncanny valley valley, where people reflexively have to find something wrong with any CGI, even to the point of imagining it, and post about it to show how smart they are.
(Score: 2) by JoeMerchant on Saturday March 24 2018, @08:05PM (1 child)
Polar Express had me fooled long enough that I was questioning whether it was live action with CGI post manipulations, for at least 30 seconds.
That feeling when you transition from being fooled to not being fooled is what makes the uncanny valley uncomfortable. There are literally millions of potential cues per minute, once enough pile up to put you in the questioning state, it's just weird there. Once you're clearly on the Pixar side of the Universe, you can get back to accepting things that talk but shouldn't, and enjoy the story.
🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Saturday March 24 2018, @08:17PM
http://i0.kym-cdn.com/photos/images/original/000/073/378/1285363898297.jpg [kym-cdn.com]
(Score: 0) by Anonymous Coward on Saturday March 24 2018, @09:00PM (1 child)
That one was pretty good. Unfortunatly for me the compression artifacts messed with the video. Lip sync was slightly off and the mouth was a bit odd. But other than that pretty good.
(Score: 2) by Rivenaleem on Monday March 26 2018, @11:00AM
"The mouth is odd" The problem with the mouth can be easily seen in the behind the scenes video, and it's mostly down to the tongue. The tongue is not easily captured, so it disappears in the digital version. In the first clip you only see a tiny bit of it for only a fraction of a second. But in the behind the scenes you can see how there's more of it visible, and light reflections off her teeth, tongue and gums/cheeks. This is all missing from the digital which makes talking models more likely to throw you off.
(Score: 0) by Anonymous Coward on Sunday March 25 2018, @01:23AM
You must mean someone who is a Pixel Pedant.
(Score: 3, Informative) by Bot on Saturday March 24 2018, @10:57PM (1 child)
count me in the unvanny valley. the face still looks too meatbaggy. Arms and dress are sexxxyyyy.
Account abandoned.
(Score: 2) by KritonK on Sunday March 25 2018, @12:03PM
Oddly enough, I found her hands to be somewhat wrong. Her face looked real to me. Too real, as it turns out, as the digital character did not have any of the skin blemishes of the original actress.
(Score: 2) by MichaelDavidCrawford on Sunday March 25 2018, @06:19AM (1 child)
... is for actors and actresses to be implemented in code so well that they don't need to employ live humans anymore.
It's not just the pay that superstars demand, but that some actors and actresses can be quite difficult to get along with.
The friend who I call "Who Moved My Cheese" was once an actress. And I have a damn good reason to choose that particular nickname.
Yes I Have No Bananas. [gofundme.com]
(Score: 3, Informative) by Wootery on Monday March 26 2018, @11:09AM
Ok, but this isn't really the time for that rant. The point here is that the video is apparently rendered in real time. Pre-rendered characters already look better than that.
(Score: 2) by KritonK on Sunday March 25 2018, @12:16PM (3 children)
So, we have a very expensive way of shooting an actress and projecting her image on screen, by digitizing her face and movements, applying these data on a digital model and rendering it in real time. I could have done this a lot cheaper by simply putting a red dress on the actress and shooting her with a camera.
Wake me up when a computer can generate all those subtle movements and expressions on its own, without needing to copy them from a human.
(Score: 2) by Wootery on Monday March 26 2018, @11:06AM (2 children)
So... not even close to the same thing, then.
So you don't want to be woken up when real-time graphics (in, say, video games) look as realistic as the video? All that tells us is that you aren't interested in computer graphics.
Fair enough, but don't pretend that's some kind of great insight into their achievement.
(Score: 2) by KritonK on Monday March 26 2018, @02:11PM (1 child)
What they did is just a very complicated way of pointing a camera at a person and reproducing what the camera "sees", given that the end result is the actress appearing on screen. The fact that they decomposed the actress into zillions of bits of information and then recomposed them, does not make it intrinsically different from directly recording the data from the camera's sensor and reproducing these data on screen, with minimal intervention. Only more complicated.
This is fine, if you want to do this the hard way, just for the sake of doing it, especially if, as you say, you are interested in computer graphics, but this isn't of much use at the moment. One might argue that this is preparation for the day when computers will actually be able to produce all those subtle human movements and gestures, without needing to copy them from a human, but this is the hard part.
(Score: 2) by Wootery on Monday March 26 2018, @02:32PM
Of course it does. You're aware that the field of computer graphics is a thing that exists, right?
It's a tech demo...
And you could have used the same nonsensical reasoning to dismiss the DOOM engine, and the Quake engine, all the way down the line to the modern day...
Are you trolling, or are you unable to comprehend that a computer graphics tech demo might, you know, demonstrate a new computer graphics technology?