SoylentNews is people
SoylentNews
from the we'll-optimize-the-computes-with-AI dept.
Facebook VP: AI has a compute dependency problem
In one of his first public speaking appearances since joining Facebook to lead its AI initiatives, VP Jérôme Pesenti expressed concern about the growing amount of compute power needed to create powerful AI systems.
“I can tell you this is keeping me up at night,” Pesenti said. “The peak compute companies like Facebook and Google can afford for an experiment, we are reaching that already.”
More software innovation will be required if artificial intelligence is to grow unhindered, he said, and optimization of hardware and software — rather than brute force compute — may be critical to AI in years ahead.
Examples of systems less reliant on compute for innovative breakthroughs include Pluribus, an AI system developed by Facebook AI Research and Carnegie Mellon University and introduced today, that can take on world-class poker players. In an article in Science, researchers said Pluribus only required $150 in cloud computing to train.
The end of Moore’s Law means the compute needed to create the most advanced AI is going up.
In fact, Pesenti cited an OpenAI analysis that found the compute necessary to create state-of-the-art systems has gone up 10 times each year since 2012.
“We still see gains with increase of compute, but the pressure from the problem is just going to become bigger,” Pesenti said. “I think we will still continue to use more compute, you will still net, but it will go slower, because you cannot keep pace with 10 times a year. That’s just not possible.”
(Score: 5, Insightful) by Mer on Monday July 15 2019, @11:22AM (3 children)
What it does expose is that AI isn't anything magical. It's just applied statistics. It's a crutch for when you don't want to build a model to solve your problem.
Can't wait for computing power to really stall so techbros that just feed data bought from social networks or state institutions into other people's algorithms get ejected from the tech field when they fail to adapt and start building solutions based on analyzing the problems themselves.
Shut up!, he explained.
(Score: 2) by takyon on Monday July 15 2019, @12:36PM (1 child)
https://ai.googleblog.com/2018/12/exploring-quantum-neural-networks.html [googleblog.com]
https://www.nextplatform.com/2019/03/05/one-step-closer-to-deep-learning-on-neuromorphic-hardware/ [nextplatform.com]
There might be a "stall", but the train ain't stopping. Quantum chips could be used for machine learning, and neuromorphic chips (best suited for scaling vertically) could be used for machine learning, if they don't get used to make artificial brains outright. Techbros gonna fuck your shit up and make Skynet real.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 3, Insightful) by Mer on Monday July 15 2019, @04:12PM
Still depressing. QC's schtick is helping statistical calculations by working with amplitudes instead of discrete values, doesn't change the fact that an AI stack without a model layer is just guessing. Adversarial patterns, or their spontaneous cousins, bad angles will keep getting you inconclusive results, you'll just get those results faster and more efficiently.
Shut up!, he explained.
(Score: 0) by Anonymous Coward on Monday July 15 2019, @01:30PM
> It's a crutch for when you don't want to build a model to solve your problem.
Thanks for bringing this up again!
I still remember reading an early paper on fuzzy logic in the 1980s, it promoted fuzzy as the second coming. After I thought about it for awhile, I realized that all the hype was a slight improvement on a bang-bang controller (like a simple thermostat). It might be suitable for some "lo-tech" systems and cases where the plant to be controlled can't be well defined, but is no substitute for control theory with a good model of the process to be controlled.