Stories
Slash Boxes
Comments

SoylentNews is people

posted by martyb on Monday July 15 2019, @04:22AM   Printer-friendly
from the we'll-optimize-the-computes-with-AI dept.

Facebook VP: AI has a compute dependency problem

In one of his first public speaking appearances since joining Facebook to lead its AI initiatives, VP Jérôme Pesenti expressed concern about the growing amount of compute power needed to create powerful AI systems.

“I can tell you this is keeping me up at night,” Pesenti said. “The peak compute companies like Facebook and Google can afford for an experiment, we are reaching that already.”

More software innovation will be required if artificial intelligence is to grow unhindered, he said, and optimization of hardware and software — rather than brute force compute — may be critical to AI in years ahead.

Examples of systems less reliant on compute for innovative breakthroughs include Pluribus, an AI system developed by Facebook AI Research and Carnegie Mellon University and introduced today, that can take on world-class poker players. In an article in Science, researchers said Pluribus only required $150 in cloud computing to train.

The end of Moore’s Law means the compute needed to create the most advanced AI is going up.

In fact, Pesenti cited an OpenAI analysis that found the compute necessary to create state-of-the-art systems has gone up 10 times each year since 2012.

“We still see gains with increase of compute, but the pressure from the problem is just going to become bigger,” Pesenti said. “I think we will still continue to use more compute, you will still net, but it will go slower, because you cannot keep pace with 10 times a year. That’s just not possible.”


Original Submission

 
This discussion has been archived. No new comments can be posted.
Display Options Threshold/Breakthrough Mark All as Read Mark All as Unread
The Fine Print: The following comments are owned by whoever posted them. We are not responsible for them in any way.
  • (Score: 4, Informative) by looorg on Monday July 15 2019, @07:41AM (1 child)

    by looorg (578) on Monday July 15 2019, @07:41AM (#867110)

    So something not possible without massive amounts of data and computing power, and wouldn't be a thing without it, is to dependent on computing power? Geee ... who could have seen that coming. Please tell us more Captain Obvious.

    "More software innovation will be required if artificial intelligence is to grow unhindered, he said, and optimization of hardware and software — rather than brute force compute — may be critical to AI in years ahead."

    So more mathematics instead of just tossing more CPU cycles at the problem then? Sounds fair. Good luck with that.

    "Analysis introduced last month found that the costs of training systems like OpenAI’s GPT-2 can exceed carbon emissions of the lifetime of five cars."

    If that is somehow an issue then perhaps they shouldn't be wasting the precious AI on trivialities but more important things. So what do they use the almighty AI for at Facebook? Oh ... Deciding what advertisements to show and figuring out if a post is hatespeech or not. That sounds like about five cars worth of emissions ... or not.

    "Analysis by Data & Society fellow and Algorithmic Accountability Act coauthor Mutale Nkonde found that Facebook AI Research currently employs 146 people, none of whom are of African descent."

    What does this have to do with anything? How many are Chinese spies? Or do they here imply that AI is computational racism? Is AI master-race-science?

    Perhaps they should just conclude with that most AI projects fail, and are complete bullshit and doesn't have a single thing to do with actual AI (whatever that is) but instead are just extensive lookup tables and tree structure calculations that are dependent on nearly worthless data to perform almost as worthless functions. Facebook should be the prime example, using it to select ads and to monitor comments.

    Starting Score:    1  point
    Moderation   +2  
       Informative=2, Total=2
    Extra 'Informative' Modifier   0  
    Karma-Bonus Modifier   +1  

    Total Score:   4  
  • (Score: 0) by Anonymous Coward on Monday July 15 2019, @07:51AM

    by Anonymous Coward on Monday July 15 2019, @07:51AM (#867113)

    "Analysis by Data & Society fellow and Algorithmic Accountability Act coauthor Mutale Nkonde found that Facebook AI Research currently employs 146 people, none of whom are of African descent."

    That's no compute dependency problem. That's a white/asian dependency problem!