SoylentNews is people
SoylentNews
This AI Creates Detailed 3D Renderings from Thousands of Tourist Photos
A team of researchers at Google have come up with a technique that can combine thousands of tourist photos into detailed 3D renderings that take you inside a scene... even if the original photos used vary wildly in terms of lighting or include other problematic elements like people or cars.
The tech is called "NeRF in the Wild" or "NeRF-W" because it takes Google Brain's Neural Radiance Fields (NeRF) technology and applies it to "unstructured and uncontrolled photo collections" like the thousands of tourist photos used to create the demo you see below[1][2], and the samples in the video above[3].
It's basically an advanced, neural network-driven interpolation that manages to include geometric info about the scene while removing 'transient occluders' like people or cars and smoothing out changes in lighting.
[1] demo1.gif (36.75 MiB)
[2] demo2.gif (35.66 MiB)
[3] YouTube video (3m42s).
NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections (arXiv:2008.02268v2 [cs.CV])
(Score: 2, Interesting) by Anonymous Coward on Sunday August 30 2020, @02:25PM (1 child)
The real advance here is to use images captured under a multitude of lighting conditions and have AI normalize them into a single textured model. This is a longstanding problem even for controlled photogrammetry where shadows become geometry and it's often faster to model from scratch than fix up the point cloud.
(Score: 0) by Anonymous Coward on Sunday August 30 2020, @04:41PM
Precisely, photogrammetry using images that were taken in close proximity has been a thing for quite a while now. If you can fly a drone around a land mark capturing images of it, you can stitch it into a surprisingly accurate 3d model. For some types of objects, the result may even be better than what can be achieved using laser based 3d scanning technology. Especially if the tones are relatively flat and you don't have the ability to dirty them up for the scan.
There's also the added level of being able to display the same image under different weather conditions which wasn't practical with the older technology unless you went back and actually recorded it under different conditions or manually want in and changed the look of the model or imported into software to do raycasting on the model.