Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Monday July 15 2019, @04:22AM   Printer-friendly
from the we'll-optimize-the-computes-with-AI dept.

Facebook VP: AI has a compute dependency problem

In one of his first public speaking appearances since joining Facebook to lead its AI initiatives, VP Jérôme Pesenti expressed concern about the growing amount of compute power needed to create powerful AI systems.

“I can tell you this is keeping me up at night,” Pesenti said. “The peak compute companies like Facebook and Google can afford for an experiment, we are reaching that already.”

More software innovation will be required if artificial intelligence is to grow unhindered, he said, and optimization of hardware and software — rather than brute force compute — may be critical to AI in years ahead.

Examples of systems less reliant on compute for innovative breakthroughs include Pluribus, an AI system developed by Facebook AI Research and Carnegie Mellon University and introduced today, that can take on world-class poker players. In an article in Science, researchers said Pluribus only required $150 in cloud computing to train.

The end of Moore’s Law means the compute needed to create the most advanced AI is going up.

In fact, Pesenti cited an OpenAI analysis that found the compute necessary to create state-of-the-art systems has gone up 10 times each year since 2012.

“We still see gains with increase of compute, but the pressure from the problem is just going to become bigger,” Pesenti said. “I think we will still continue to use more compute, you will still net, but it will go slower, because you cannot keep pace with 10 times a year. That’s just not possible.”


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 2) by goodie on Monday July 15 2019, @03:50PM

    by goodie (1877) on Monday July 15 2019, @03:50PM (#867219) Journal

    Thank you Captain Obvious! More seriously, I don't see what the big news is here. I think that arguments about this from an environmental impact/sustainability standpoint are extremely worrisome. But that's not what this is about at all, it's about selling stuff...

    To me, there are 2 issues. First, when you use the cloud, you don't know how much power you are actually using to do something. You want it quick so you jack up the computing power, screw the extra costs, you have "AI" in your project name so the bosses are happy to pay! Second, scaleability encourages laziness. A lot of the stuff we see, I surmise, is not coded to optimize computing/storage resources, but rather to do as much as possible without thinking about the resources required to do it. Where the sh*t hits the fan is when you need to pack that into a small component with limited hardware capabilities. There, the scaling cannot come by adding more computing resources, it must come from using alternate models etc. But that's only doable if there are other ways to get the same outcome. In some ways, it's the same argument as the blockchain hashing computations that become incredibly hungry over time.

    Switching back to my original concern about sustainability: If only we could sit for a moment and think about *what* many AI projects are using those resources for, we may be able to really cut down. deepfakes, advertising, etc. are all fun and stuff, but they are zero value-added for us as humans and more or a negative value-added IMHO. Stuff like medicine etc. I think is much more valuable. So if we were to consider our computing resources as finite, then we would maybe think about allocating those resources better.

    And finally... Maybe we ought to think about the trade-offs we want to make between efficiency and effectiveness. In any AI/machine learning "system", you need to decide whether gaining that extra 0.01% is worth the trouble. In many situations it is not. In others (e.g., medicine again), it might make a lot of sense. Right now, in many projects, it's about who can make a framework that scales across thousands of CPUs/GPUs in a way that is transparent for the developer. But none of those are questioning the actual model, they are just writing software on top to make it scale.

    Anyway I might be way off on all this, but all I can imagine is data centers heating up to customize a news feed somewhat on some person's machine who doesn't even give a heck about it. It's Monday, I feel cynical ;(.

    Starting Score:    1  point
    Karma-Bonus Modifier   +1  

    Total Score:   2