Stories
Slash Boxes
Comments

SoylentNews is people

posted by Fnord666 on Thursday June 21 2018, @12:46AM   Printer-friendly
from the approaching-the-singularity dept.

IBM researchers use analog memory to train deep neural networks faster and more efficiently

Deep neural networks normally require fast, powerful graphical processing unit (GPU) hardware accelerators to support the needed high speed and computational accuracy — such as the GPU devices used in the just-announced Summit supercomputer. But GPUs are highly energy-intensive, making their use expensive and limiting their future growth, the researchers explain in a recent paper published in Nature.

Instead, the IBM researchers used large arrays of non-volatile analog memory devices (which use continuously variable signals rather than binary 0s and 1s) to perform computations. Those arrays allowed the researchers to create, in hardware, the same scale and precision of AI calculations that are achieved by more energy-intensive systems in software, but running hundreds of times faster and at hundreds of times lower power — without sacrificing the ability to create deep learning systems.

The trick was to replace conventional von Neumann architecture, which is "constrained by the time and energy spent moving data back and forth between the memory and the processor (the 'von Neumann bottleneck')," the researchers explain in the paper. "By contrast, in a non-von Neumann scheme, computing is done at the location of the data [in memory], with the strengths of the synaptic connections (the 'weights') stored and adjusted directly in memory.

Equivalent-accuracy accelerated neural-network training using analogue memory (DOI: 10.1038/s41586-018-0180-5) (DX)


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by RS3 on Thursday June 21 2018, @01:56AM (10 children)

    by RS3 (6367) on Thursday June 21 2018, @01:56AM (#695931)

    Everyone has been led to believe that digital is the end-all. We will be owned by analog bots.

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2  
  • (Score: 2) by bob_super on Thursday June 21 2018, @02:12AM

    by bob_super (1357) on Thursday June 21 2018, @02:12AM (#695944)

    We're currently owned by organic bots, after all.

  • (Score: 0) by Anonymous Coward on Thursday June 21 2018, @02:53AM (8 children)

    by Anonymous Coward on Thursday June 21 2018, @02:53AM (#695967)

    Unless you have a way of digitizing molecular structures then duh.

    • (Score: 2) by c0lo on Thursday June 21 2018, @04:03AM (6 children)

      by c0lo (156) Subscriber Badge on Thursday June 21 2018, @04:03AM (#696000) Journal

      Unless you have a way of digitizing molecular structures then duh.

      Given the discrete QM nature of the molecular structures, we are already there.
      A pity the same QM nature doesn't (yet) allow us to control the so many discrete states those structures can exhibit.

      --
      https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
      • (Score: 2) by RS3 on Thursday June 21 2018, @04:07AM (1 child)

        by RS3 (6367) on Thursday June 21 2018, @04:07AM (#696002)

        D'oh! You beat me by 2 minutes.

        • (Score: 2) by c0lo on Thursday June 21 2018, @04:30AM

          by c0lo (156) Subscriber Badge on Thursday June 21 2018, @04:30AM (#696020) Journal

          Sorry, I just didn't realize there was a competition going.

          --
          https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
      • (Score: 2) by RS3 on Thursday June 21 2018, @04:22AM (3 children)

        by RS3 (6367) on Thursday June 21 2018, @04:22AM (#696014)

        A pity the same QM nature doesn't (yet) allow us to control the so many discrete states those structures can exhibit.

        I'm curious what you mean. I believe many things function by stimulating energy states, such as pretty much anything that gives off photons / light including LASERs, magnetrons, klystrons, TWAT, vacuum tubes (electronic "valves"), X-ray, etc. So I'm guessing you mean some more advanced thing?

        • (Score: 2) by c0lo on Thursday June 21 2018, @04:36AM (2 children)

          by c0lo (156) Subscriber Badge on Thursday June 21 2018, @04:36AM (#696025) Journal

          In all examples, you take advantages of a large set of molecular structures so that you apply sorta "macroscopic" filter over the output and discard the rest of the output. This doesn't mean control, it just mean "selection".

          We aren't in the technological position to control the states of a molecular structures at individual levels - and Heisenberg warned us again thinking that we'll ever be able.

          --
          https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
          • (Score: 2) by RS3 on Thursday June 21 2018, @06:14AM (1 child)

            by RS3 (6367) on Thursday June 21 2018, @06:14AM (#696056)

            Okay, thanks. But aren't narrow spectrum light sources doing this?

            But generally your answer is what I envisioned you were alluding to, at the most advanced concept. So one electric field exciting wire per atom. Or maybe a very focused electron beam. Hey, don't scanning tunneling electron microscopes approach that?

            So what would be the purpose, output, whatever, of doing it? New molecules and compounds? Superconductivity? Something relating to nuclear fusion? Gene editing? Medicines? Tricorders? All of the above and more?

            • (Score: 2) by c0lo on Thursday June 21 2018, @06:45AM

              by c0lo (156) Subscriber Badge on Thursday June 21 2018, @06:45AM (#696066) Journal

              Hey, don't scanning tunneling electron microscopes approach that?

              Measuring != controlling

              So what would be the purpose, output, whatever, of doing it?

              CPU at Ångström-unit scale? :)

              --
              https://www.youtube.com/watch?v=aoFiw2jMy-0 https://soylentnews.org/~MichaelDavidCrawford
    • (Score: 2) by RS3 on Thursday June 21 2018, @04:05AM

      by RS3 (6367) on Thursday June 21 2018, @04:05AM (#696001)

      Pretty sure molecular structures are held together by bonds that obey laws described by quantum mechanics. But I'm not a chemist nor physicist. Well, a little.