SoylentNews is people
SoylentNews
IBM researchers use analog memory to train deep neural networks faster and more efficiently
Deep neural networks normally require fast, powerful graphical processing unit (GPU) hardware accelerators to support the needed high speed and computational accuracy — such as the GPU devices used in the just-announced Summit supercomputer. But GPUs are highly energy-intensive, making their use expensive and limiting their future growth, the researchers explain in a recent paper published in Nature.
Instead, the IBM researchers used large arrays of non-volatile analog memory devices (which use continuously variable signals rather than binary 0s and 1s) to perform computations. Those arrays allowed the researchers to create, in hardware, the same scale and precision of AI calculations that are achieved by more energy-intensive systems in software, but running hundreds of times faster and at hundreds of times lower power — without sacrificing the ability to create deep learning systems.
The trick was to replace conventional von Neumann architecture, which is "constrained by the time and energy spent moving data back and forth between the memory and the processor (the 'von Neumann bottleneck')," the researchers explain in the paper. "By contrast, in a non-von Neumann scheme, computing is done at the location of the data [in memory], with the strengths of the synaptic connections (the 'weights') stored and adjusted directly in memory.
Equivalent-accuracy accelerated neural-network training using analogue memory (DOI: 10.1038/s41586-018-0180-5) (DX)
(Score: 1, Informative) by Anonymous Coward on Thursday June 21 2018, @02:37AM (1 child)
Weren't Perceptrons analog?
[googles]
Here's one paper from 2013 https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3574673/ [nih.gov]
> This study examines an analog circuit comprising a multilayer perceptron neural network (MLPNN). This study proposes a low-power and small-area analog MLP circuit to implement in an E-nose as a classifier, such that the E-nose would be relatively small, power-efficient, and portable. The analog MLP circuit had only four input neurons, four hidden neurons, and one output neuron. The circuit was designed and fabricated using a 0.18 μm standard CMOS process with a 1.8 V supply. The power consumption was 0.553 mW, and the area was approximately 1.36 × 1.36 mm2. The chip measurements showed that this MLPNN successfully identified the fruit odors of bananas, lemons, and lychees with 91.7% accuracy.
Marvin Minsky debunked Perceptrons with his famous book of the same name, but now it's looking more like the only problem with the early attempts was that they weren't big enough and/or deep enough.
(Score: 2) by fritsd on Thursday June 21 2018, @04:56PM
Minsky and Papert spotted that, if you use a linear activation function, then whatever depth of Perceptron can be re-written as a simple multivariate linear equation.
It's a pity that many people then gave up on Perceptrons until Rumelhart and McClelland (IIRC) rekindled the interest a long time after.