Facebook VP: AI has a compute dependency problem
In one of his first public speaking appearances since joining Facebook to lead its AI initiatives, VP Jérôme Pesenti expressed concern about the growing amount of compute power needed to create powerful AI systems.
“I can tell you this is keeping me up at night,” Pesenti said. “The peak compute companies like Facebook and Google can afford for an experiment, we are reaching that already.”
More software innovation will be required if artificial intelligence is to grow unhindered, he said, and optimization of hardware and software — rather than brute force compute — may be critical to AI in years ahead.
Examples of systems less reliant on compute for innovative breakthroughs include Pluribus, an AI system developed by Facebook AI Research and Carnegie Mellon University and introduced today, that can take on world-class poker players. In an article in Science, researchers said Pluribus only required $150 in cloud computing to train.
The end of Moore’s Law means the compute needed to create the most advanced AI is going up.
In fact, Pesenti cited an OpenAI analysis that found the compute necessary to create state-of-the-art systems has gone up 10 times each year since 2012.
“We still see gains with increase of compute, but the pressure from the problem is just going to become bigger,” Pesenti said. “I think we will still continue to use more compute, you will still net, but it will go slower, because you cannot keep pace with 10 times a year. That’s just not possible.”
(Score: 2) by legont on Monday July 15 2019, @06:18PM (1 child)
Yep. In fact practical neural networks were developed in late 70s and I had my degree project on some now forgotten flavor back in 1981. It was supposed to greatly reduce compute required, believe it or not.
"Wealth is the relentless enemy of understanding" - John Kenneth Galbraith.
(Score: 0) by Anonymous Coward on Monday July 15 2019, @09:52PM
It's entirely possible that neural networks will have some kind of viable use as a structural component of a functional AI architecture, doing specific jobs, but the current approach of MOAR NEURONZ ain't getting us there.