SoylentNews is people
SoylentNews
Submission Preview
Scientists claim broadcast quality 2-D to 3-D video conversion from playing video game
MIT News [mit.edu] says researchers at MIT and the Qatar Computing Research Institute (QCRI) have developed a system that automatically converts 2-D video of soccer to 3-D video that can be played on any 3-D device.
Unfortunately, their system only works for soccer video and relies on the existence of a soccer video game.
“We are developing a conversion pipeline for a specific sport. We would like to do it at broadcast quality, and we would like to do it in real-time. What we have noticed is that we can leverage video games.”
Today’s video games generally store very detailed 3-D maps of the virtual environment that the player is navigating. When the player initiates a move, the game adjusts the map accordingly and, on the fly, generates a 2-D projection of the 3-D scene that corresponds to a particular viewing angle.
The MIT and QCRI researchers essentially ran this process in reverse. They set the very realistic Microsoft soccer game “FIFA13” to play over and over again, and used Microsoft’s video-game analysis tool PIX to continuously store screen shots of the action. For each screen shot, they also extracted the corresponding 3-D map.
[...After] keeping just those [images] that best captured the range of possible viewing angles and player configurations that the game presented...they stored each screen shot and the associated 3-D map in a database.
[...]For every frame of 2-D video of an actual soccer game, the system looks for the 10 or so screen shots in the database that best correspond to it. Then it decomposes all those images, looking for the best matches between smaller regions of the video feed and smaller regions of the screen shots. Once it’s found those matches, it superimposes the depth information from the screen shots on the corresponding sections of the video feed. Finally, it stitches the pieces back together.
The result is a very convincing 3-D effect, with no visual artifacts. The researchers conducted a user study in which the majority of subjects gave the 3-D effect a rating of 5 (“excellent”) on a five-point (“bad” to “excellent”) scale; the average score was between 4 (“good”) and 5.
