Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Saturday July 04 2020, @03:10PM   Printer-friendly
from the making-a-mountain-out-of-a-mole-hill dept.

Neural SuperSampling Is a Hardware Agnostic DLSS Alternative by Facebook

A new paper published by Facebook researchers just ahead of SIGGRAPH 2020 introduces neural supersampling, a machine learning-based upsampling approach not too dissimilar from NVIDIA's Deep Learning Super Sampling. However, neural supersampling does not require any proprietary hardware or software to run and its results are quite impressive as you can see in the example images, with researchers comparing them to the quality we've come to expect from DLSS.

Video examples on Facebook's blog post.

The researchers use some extremely low-fi upscales to make their point, but you could also imagine scaling from a resolution like 1080p straight to 8K. Upscaling could be combined with eye tracking and foveated rendering to reduce rendering times even further.

Also at UploadVR and VentureBeat.

Journal Reference:
Lei Xiao, Salah Nouri, Matt Chapman, Alexander Fix, Douglas Lanman, Anton Kaplanyan,Neural Supersampling for Real-time Rendering - Facebook Research, (DOI: https://research.fb.com/publications/neural-supersampling-for-real-time-rendering/)

Related: With Google's RAISR, Images Can be Up to 75% Smaller Without Losing Detail
Nvidia's Turing GPU Pricing and Performance "Poorly Received"
HD Emulation Mod Makes "Mode 7" SNES Games Look Like New
Neural Networks Upscale Film From 1896 to 4K, Make It Look Like It Was Shot on a Modern Smartphone
Apple Goes on an Acquisition Spree, Turns Attention to NextVR


Original Submission

Related Stories

With Google's RAISR, Images Can be Up to 75% Smaller Without Losing Detail 26 comments

With unlimited data plans becoming increasingly expensive, or subscribers being forced to ditch their unlimited data due to overuse, anything that can reduce the amount of data we download is welcome. This is especially true for media including images or video, and Google just delivered a major gain when it comes to viewing images online.

The clever scientists at Google Research have come up with a new technique for keeping image size to an absolute minimum without sacrificing quality. So good is this new technique that it promises to reduce the size of an image on disk by as much as 75 percent.

The new technique is called RAISR, which stands for "Rapid and Accurate Image Super-Resolution." Typically, reducing the size of an image means lowering its quality or resolution. RAISR works by taking a low-resolution image and upsampling it, which basically means enhancing the detail using filtering. Anyone who's ever tried to do this manually knows that the end result looks a little blurred. RAISR avoids that thanks to machine learning.

[...] RAISR has been trained using low and high quality versions of images. Machine learning allows the system to figure out the best filters to recreate the high quality image using only the low quality version. What you end up with after lots of training is a system that can do the same high quality upsampling on most images without needing the high quality version for reference.

-- submitted from IRC


Original Submission

Nvidia's Turing GPU Pricing and Performance "Poorly Received" 20 comments

Nvidia's Turing pricing strategy has been 'poorly received,' says Instinet

Instinet analyst Romit Shah commented Friday on Nvidia Corp.'s new Turing GPU, now that reviews of the product are out. "The 2080 TI is indisputably the best consumer GPU technology available, but at a prohibitive cost for many gamers," he wrote. "Ray tracing and DLSS [deep learning super sampling], while apparently compelling features, are today just 'call options' for when game developers create content that this technology can support."

Nvidia shares fall after Morgan Stanley says the performance of its new gaming card is disappointing

"As review embargos broke for the new gaming products, performance improvements in older games is not the leap we had initially hoped for," Morgan Stanley analyst Joseph Moore said in a note to clients on Thursday. "Performance boost on older games that do not incorporate advanced features is somewhat below our initial expectations, and review recommendations are mixed given higher price points." Nvidia shares closed down 2.1 percent Thursday.

Moore noted that Nvidia's new RTX 2080 card performed only 3 percent better than the previous generation's 1080Ti card at 4K resolutions.

HD Emulation Mod Makes “Mode 7” SNES Games Look Like New 7 comments

Submitted via IRC for AzumaHazuki

HD emulation mod makes "Mode 7" SNES games look like new

Gamers of a certain age probably remember being wowed by the quick, smooth scaling and rotation effects of the Super Nintendo's much-ballyhooed "Mode 7" graphics. Looking back, though, those gamers might also notice how chunky and pixelated those background transformations could end up looking, especially when viewed on today's high-end screens.

Emulation to the rescue. A modder going by the handle DerKoun has released an "HD Mode 7" patch for the accuracy-focused SNES emulator bsnes. In their own words, the patch "performs Mode 7 transformations... at up to 4 times the horizontal and vertical resolution" of the original hardware.

[...] Games that made use of the SNES "Graphics Mode 7" used backgrounds that were coded in the SNES memory as a 128x128 grid of 256-color, 8x8 pixel tiles. That made for a 1024×1024 "map" that could be manipulated en masse by basic linear algebra affine transforms to rotate, scale, shear, and translate the entire screen quickly.

Some Mode 7 games also made use of an additional HDMA mode (Horizontal-blanking Direct Memory Access) to fake a "3D" plane that stretches off into the horizon. These games would essentially draw every horizontal scanline in a single SDTV frame at a different scale, making pieces lower in the image appear "closer" than ones far away.

It's a clever effect but one that can make the underlying map data look especially smeary and blob-like, especially for parts of the map that are "far away." This smearing is exacerbated by the SNES' matrix math implementation, which uses trigonometric lookup tables and rounding to cut down on the time needed to perform all that linear algebra on '90s-era consumer hardware. Translating those transformation results back to SNES-scale tiles and a 420p SD screen leads to some problems on the edges of objects, which can look lumpy and "off" by a pixel or two at certain points on the screen.

The HD Mode 7 mod fixes this problem by making use of modern computer hardware to perform its matrix math "at the output resolution," upscaling the original tiles before any transformations are done. This provides more accurate underlying "sub-pixel" data, which lets the emulator effectively use the HD display and fill in some of the spaces between those "boxy" scaled-up pixels.


Original Submission

Neural Networks Upscale Film From 1896 to 4K, Make It Look Like It Was Shot on a Modern Smartphone 26 comments

Neural Networks Upscale Film from 1896 to 4K, Make It Look Like It Was Shot on a Modern Smartphone:

When watching old film footage that's plagued with excessive amounts of grain, gate weave, soft focus, and a complete lack of color, it's hard to feel connected to the people in the clip, or what's going on. It looks like a movie, and over the years that medium has taught our brains that what they're seeing on screen might not actually be real. By comparison, the experience of watching videos of friends and family captured on your smartphone is completely different thanks to 4K resolutions and high frame-rates. Those clips feel more authentic and while watching them there's more of a connection to the moment, even if you weren't actually there while it was being shot.

[...] L'Arrivée d'un train en gare de La Ciotat doesn't have the same effect on modern audiences, but Denis Shiryaev wondered if it could be made more compelling by using neural network powered algorithms (including Topaz Labs' Gigapixel AI and DAIN) to not only upscale the footage to 4K, but also increase the frame rate to 60 frames per second. You might yell at your parents for using the motion smoothing setting on their fancy new TV, but here the increased frame rate has a dramatic effect on drawing you into the action.

[...] The results are far from perfect; we're hoping Shiryaev applies one of the many deep learning algorithms that can colorize black and white photos to this film as well, but the obvious potential of these tools to enhance historical footage to increase its impact is just as exciting as the potential for it to be misused.


Original Submission

Apple Goes on an Acquisition Spree, Turns Attention to NextVR 7 comments

Exclusive: Apple likely buyer of NextVR, a live event streaming AR/VR company being sold for ~$100M

It's no secret that Apple has ambitious plans for augmented reality and a future AR-focused headset. Apple is practically building the platform for its future headset out in the open with ARKit. What's new is that Apple is believed to be in the process of acquiring a California-based virtual reality company called NextVR, 9to5Mac has learned.

NextVR, which is located in Orange County, California, has a decade of experience marrying virtual reality with sports and entertainment. The company currently provides VR experiences for viewing live events with headsets from PlayStation, Oculus, HTC, Microsoft, Lenovo headsets.

The icing on the cake may not be expertise in virtual reality, however, as NextVR also has holds patented technology that upscales video streams. NextVR uses this technology to support high quality video streams of music and sporting events to VR headsets. NextVR holds over 40 technology patents in total.

Apple is reportedly in the process of snapping up NextVR, its third acquisition in the past week

Apple appears to have embarked on a buying spree over the past week, as startup valuations come down amid the coronavirus pandemic.

[...] Apple frequently buys smaller startups without disclosing the details, Apple CEO Tim Cook told CNBC last May. But an uptick in acquisitions — three in a week — is particularly significant as startups tackle the economic pressures brought by the coronavirus pandemic.

Last week, Apple acquired the acclaimed weather app DarkSky, in a move predicted to add to a growing list of services division. DarkSky's founder Adam Grossman announced the news in a blog post, but didn't disclose any of the deal's details.

Then it acquired the Dublin-based AI startup Voysis, whose technology could help bolster Siri's language skills, according to Bloomberg's Mark Gurman. The terms of the deal were also left undisclosed.


Original Submission

This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
(1)
  • (Score: 0) by Anonymous Coward on Saturday July 04 2020, @03:35PM (4 children)

    by Anonymous Coward on Saturday July 04 2020, @03:35PM (#1016141)

    Great, now you can try to blur out people you want unrecognizable and it'll bloody well undo that blur... Thanks In-q-tel...

    • (Score: 2) by takyon on Saturday July 04 2020, @03:43PM (2 children)

      by takyon (881) <takyonNO@SPAMsoylentnews.org> on Saturday July 04 2020, @03:43PM (#1016143) Journal

      If you want to make people unrecognizable, you need to cover them up with a single solid color.

      --
      [SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
      • (Score: 0) by Anonymous Coward on Saturday July 04 2020, @03:55PM

        by Anonymous Coward on Saturday July 04 2020, @03:55PM (#1016149)

        >> you need to cover them up with a single solid color.

        But not black, which AI trained on current datasets recognizes as criminal.

      • (Score: 1) by anubi on Saturday July 04 2020, @11:25PM

        by anubi (2828) on Saturday July 04 2020, @11:25PM (#1016305) Journal

        Cut and paste a blur from someone else. Public figures.

        --
        "Prove all things; hold fast that which is good." [KJV: I Thessalonians 5:21]
    • (Score: 2) by rleigh on Sunday July 05 2020, @10:07AM

      by rleigh (4887) on Sunday July 05 2020, @10:07AM (#1016464) Homepage

      You can do that without any machine learning (within certain limits). All the blurring has done is spread out the signal over a wide range of pixels. It doesn't take much effort to gather it back in. Running a convolution kernel over the blurred region is often sufficient to reconstruct a very close approximation to the original.

  • (Score: 0) by Anonymous Coward on Saturday July 04 2020, @09:42PM (2 children)

    by Anonymous Coward on Saturday July 04 2020, @09:42PM (#1016265)

    Now that would be a technology worth developing.

  • (Score: 2) by rleigh on Sunday July 05 2020, @10:18AM (1 child)

    by rleigh (4887) on Sunday July 05 2020, @10:18AM (#1016466) Homepage

    The problem with all of these "machine learning" reconstruction techniques lies in the assumptions they make. The trained model might be able to do a good job on the images it has been trained for, but this doesn't mean it can do a good job for any other images. What's shown here is truly impressive, but one does wonder where all the extra detail came from if it wasn't present in the source image(s).

    Take the image later in the article with the letters "SH" on the wall. The input image is just a handful of spotty red pixels. There's no way you can use a single frame to extrapolate the precise lines of the text. Looking at the video, the aliased text in the low resolution image over time might allow such a reconstruction, but the upsampled copy is good from the first frame on. I'm a bit sceptical of just how many baked in assumptions there are in the model to allow it to do that. Same applies to the serifs on the text in the first image as well as the very detailed textures on the floor and the couch.

    The issue I have with much of this is it's difficult to tell the difference between "reconstruction" and "invention". How do we tell the difference between an accurate reconstruction of the original image, and mere filling in with invented detail. For artistic purposes like games, it's not overly important. But for medical imaging it really does matter. But people are trying to use it for that type of purpose, despite the strong possibility that it's largely fictional. The same applies to using machine learning for medical diagnoses. You cannot invent details which are not present in the source image.

    • (Score: 2) by takyon on Sunday July 05 2020, @12:15PM

      by takyon (881) <takyonNO@SPAMsoylentnews.org> on Sunday July 05 2020, @12:15PM (#1016481) Journal

      The target of this paper is VR headsets, specifically Oculus which Facebook owns.

      For medical imaging, does "inventing" details help the algorithm get better detection rates and lower false positives? If not, then it's not an improvement.

      --
      [SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(1)