Tech Review reports on a study of the energy (carbon) costs of training an AI to do natural language processing and compares to the lifecycle costs of cars,
https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/
In a new paper, researchers at the University of Massachusetts, Amherst, performed a life cycle assessment for training several common large AI models. They found that the process can emit more than 626,000 pounds of carbon dioxide equivalent—nearly five times the lifetime emissions of the average American car (and that includes manufacture of the car itself).
It’s a jarring quantification of something AI researchers have suspected for a long time. “While probably many of us have thought of this in an abstract, vague level, the figures really show the magnitude of the problem,” says Carlos Gómez-Rodríguez, a computer scientist at the University of A Coruña in Spain, who was not involved in the research. “Neither I nor other researchers I’ve discussed them with thought the environmental impact was that substantial.”
In the grand scheme of things, five cars out of the millions made every year isn't a very big deal...but your faithful AC would never have guessed that it took anywhere near that much energy.
Related Stories
The Association for Computing Machinery has a post by George Neville-Neil of FreeBSD fame comparing LLMs to drunken plagiarists:
Before trying to use these tools, you need to understand what they do, at least on the surface, since even their creators freely admit they do not understand how they work deep down in the bowels of all the statistics and text that have been scraped from the current Internet. The trick of an LLM is to use a little randomness and a lot of text to Gauss the next word in a sentence. Seems kind of trivial, really, and certainly not a measure of intelligence that anyone who understands the term might use. But it's a clever trick and does have some applications.
[...] While help with proper code syntax is a boon to productivity (consider IDEs that highlight syntactical errors before you find them via a compilation), it is a far cry from SEMANTIC knowledge of a piece of code. Note that it is semantic knowledge that allows you to create correct programs, where correctness means the code actually does what the developer originally intended. KV can show many examples of programs that are syntactically?but not semantically?correct. In fact, this is the root of nearly every security problem in deployed software. Semantics remains far beyond the abilities of the current AI fad, as is evidenced by the number of developers who are now turning down these technologies for their own work.
He continues by pointing out how LLMs are not only based on plagiarism, they are unable provide useful annotation in the comments or otherwise address the semantics of the code they swipe.
Previously:
(2024) Make Illegally Trained LLMs Public Domain as Punishment
(2024) The Open Secret Of Open Washing
(2023) A Jargon-Free Explanation of How AI Large Language Models Work
(2019) AI Training is *Very* Expensive
... and many more.
(Score: 2) by krishnoid on Friday June 07 2019, @08:54AM (6 children)
It's interesting that this was a blind spot for so many people who are stereotypically very detail-oriented. I suspect the people who design PC power supplies already knew this, though.
(Score: 1, Insightful) by Anonymous Coward on Friday June 07 2019, @09:31AM (4 children)
"That number was converted into pounds of carbon dioxide equivalent based on the average energy mix in the US, which closely matches the energy mix used by Amazon’s AWS, the largest cloud services provider."
Erm.
So first, AWS is worldwide. Secondly, training an AI doesn't mean you're going to be using AWS.
So, it really matters *where* you train. And even what hardware you use. For example, here in Quebec there is *loads* of excess, and I mean excess (not used) power much of the year. (After all, how can you power homes heating at -40C in Feb, if you don't have excess capacity when it's 20C out in June?).
So.. if you're using hardware in Quebec, and OVH (for example) has massive infrastructure here, then you're really almost at zero for carbon usage. At least compared to the "US mix" used in the article.. which cites coal as power stations...
More than anything, this is almost a political statement about 'source of power', and 'location, location, location!'.. rather than 'this is environmentally costly'.
(Score: 1, Insightful) by Anonymous Coward on Friday June 07 2019, @11:11AM (3 children)
Naive assessment. Their approach is realistic, your approach is naive. Like saying "I don't have to worry about global warming, because my house is 100% solar powered". But reality is, you are not an island.
(Score: 1, Touché) by Anonymous Coward on Friday June 07 2019, @11:47AM
You just went on about 'not being an island', yet did not describe how, in this case, Quebec is not.
However, Quebec is a massive exporter of power. It has *immense* idle and fallow power generation at specific times of year, which I delineated in my initial post.
Of course, you didn't respond to *that* part of my statement, nor to the concept that there are 'dirty' parts of the world, and clean, to generate power.
Nope. Just a retort with no counter points, no data, no information, and pithy comments.
(Score: 1, Informative) by Anonymous Coward on Friday June 07 2019, @11:56AM (1 child)
I fact, reading your reply again, I don't believe you "get" it.
My point is that if people used power-clean areas to compute, then this study is moot. And that this study is referencing only a very dirty area, and even claiming AWS is ONLY in that area.
It isn't.
Therefore, if someone sets up in Quebec? This study is entirely null and void. I'm meaningless. And it's meaningless, because their 'conclusion' doesn't use a metric that "makes sense".
It's like saying "It costs $1000 to cool a house!", on a study referencing worldwide housing... uh, no. Location counts, you *can not* make location specific statements, to non-location specific activities.
(Score: 3, Funny) by fyngyrz on Friday June 07 2019, @02:59PM
Absolutely. Our local power is hydro. [wikipedia.org] Plus we have a solar installation. I really don't worry that I'm polluting due to my electrical usage.
Also, for many AI systems, the training is done once at some notable energy cost, then the already-trained kernel is distributed to users where each instance is a near-negligible user of power. That's kind of the point of most such systems anyway. At that juncture, they're doing yeoman work for very little cost.
So unless the training is being done for every use... the article is more than a little misleading.
Also, just a little perspective:
It's always worth noting that the human body, at rest, continuously uses about 60 watts. We're more significant users of power (albeit derived from plants and animals, mostly) than 100 wall warts that are just sitting there without actually charging or running anything — when a device is just barely CEC certified (0.5W no-load dissipation), which is not at all uncommon. And of course, many times wall warts are charging/running something, and accordingly consume much more power.
So hey, if you want to be ecologically responsible, consider reducing the number of Republicans. Two birds, one stone. 😊
--
Things aren't always #000000 and #FFFFFF.
(Score: 1) by khallow on Friday June 07 2019, @08:21PM
(Score: 2) by PiMuNu on Friday June 07 2019, @10:17AM (5 children)
> In a new paper, researchers at the University of Stupidity, performed a life cycle assessment for several common equipment binding solutions. They found that the solutions can require more than 626,000 metres of length -
> nearly 626,000 times the length of other common equipment binding solutions..
>
> It’s a jarring quantification of something binding researchers have suspected for a long time. “While probably many of us have thought of this in an abstract, vague level, the figures really show the magnitude of the problem,”
> says Carlos Stupid-Stupid, a string theorist at the University of Stupido in Spain, who was not involved in the research. “Neither I nor other researchers I’ve discussed them with thought the environmental impact was that
> substantial.”
(Score: 2) by PiMuNu on Friday June 07 2019, @10:19AM (4 children)
> 626,000 metres
Sorry I made a typo, it was 626,000 yards.
(Score: 2) by Pslytely Psycho on Friday June 07 2019, @10:47AM
Careful now, it's mistakes like this that cost Orbiters.....(:
Alex Jones lawyer inspires new TV series: CSI Moron Division.
(Score: 2) by VLM on Friday June 07 2019, @12:06PM (2 children)
The numbers are probably real.
When journalists specify 656167.9 yards with seven sig figs you know its because some engineer answering a question acknowledged he only has like 1 sig fig of accuracy so he off the cuff estimated "aw heck I donno maybe six hundred kilometers or so" and the journalist did meters to yards and upgraded the sig figs from about one to seven.
The proposed numbers mentioned are not magic nor suspiciously rounded, unlike my example, so they're probably real trustworthy numbers.
(Score: 3, Interesting) by PiMuNu on Friday June 07 2019, @12:49PM (1 child)
I don't doubt, that if one runs a cluster for $X amount of time, one can rack up $Y carbon footprint. But $X varies wildly depending on the problem, making the study worthless. That is why I compared it to "length of a piece of string" and indicated a certain contempt for the authors of the paper.
(Score: 2) by VLM on Saturday June 08 2019, @03:45PM
We are not quite in the era of engineered AI where we can predict the costs.
Compare to something like a railroad line even in the old days it was possible to engineer something accurately predicting cost enough to keep the beancounters at bay.
Possibly AI wrangling is going to be like programming where its not really possible to estimate costs unlike real engineering disciplines.
(Score: 4, Informative) by JoeMerchant on Friday June 07 2019, @11:09AM (15 children)
So, it takes a lot of energy to fully train a natural language processing neural net, emitting as much CO2 as (GASP!) 5 cars over their lifetime - call that 1 million road miles.
But, once trained, what's the value of this AI? Can it reduce staff in a call center? Can it reduce staff in a thousand call centers? How much CO2 do we emit "training" just one BI (biological intelligence) to staff a call center 8 hours a day - and don't forget the coffee breaks, health insurance, and CO2 cost of feeding and housing this BI while it works.
If you want to get pissed about CO2 emissions for compute cycles of questionable value, get pissed at proof-of-work cryptocurrencies, particularly the most popular ones. Compute cycles that put people out of work? If they're out of the lab and being put to use in the real world, that's a net win.
Oh, and if you want to reduce my CO2 emissions, just give me a hut with a garden plot and some goats on a South Pacific island, and a UBI check to cover miscellaneous expenses - I'll give up the cars, daily commute and grocery store, thanks.
🌻🌻🌻 [google.com]
(Score: 2) by shortscreen on Friday June 07 2019, @01:06PM (12 children)
You are assuming that former call center personel who are replaced by AI simply cease to exist. Maybe where you live, laid-off workers are executed and recycled into soylent or something, but that surely isn't the norm.
(Score: 3, Informative) by JoeMerchant on Friday June 07 2019, @01:20PM (11 children)
A) Population control is coming, one way or another.
B) As part of "population" I'd much rather add value to society just about any way OTHER than working for a call center, or nearly any other job that today's AI can be trained to do.
C) UBI is the answer.
🌻🌻🌻 [google.com]
(Score: 3, Touché) by EvilSS on Friday June 07 2019, @01:55PM
If UBIsoft if the answer, we're all doomed.
(Score: 2) by Freeman on Friday June 07 2019, @02:50PM (8 children)
The United States of America is 131st on the list of population growth rate, Germany is 164th, South Korea is 151st, Switzerland is 122nd, the UK is 136th, and Canada is 118th.
https://en.wikipedia.org/wiki/List_of_countries_by_population_growth_rate [wikipedia.org]
Now, compare that to this listing:
"... gross domestic product (at purchasing power parity) per capita, i.e., the purchasing power parity (PPP) value of all final goods and services produced within a country in a given year, divided by the average (or mid-year) population for the same year."
The United States of America is 13th on the list of countries by GDP (PPP), Germany is 19th, South Korea is 32nd, Switzerland is 11th, the UK is 29th, and Canada is 24th.
https://en.wikipedia.org/wiki/List_of_countries_by_GDP_(PPP)_per_capita [wikipedia.org]
I picked those countries at random as countries that I would consider to be doing pretty well. When you say "Population control is coming", you're saying those people in Africa should stop having kids.
Compare this map of Population Growth Rate:
https://en.wikipedia.org/wiki/File:Population_growth_rate_world_2005-2010_UN.PNG [wikipedia.org]
With this map of GDP(PPP):
https://en.wikipedia.org/wiki/File:Countries_by_GDP_(PPP)_per_capita_in_2017.png [wikipedia.org]
The poorest countries in the world are also, the fastest growing countries in the world.
The people touting "Population Control" aren't presenting their argument in good faith. There's pretty much no reason to be alarmed as far as prosperous nations are concerned. When you start saying, we need to control how other countries are growing at an alarming rate, then you've got an entirely different problem. Unless, you're okay with mass-murder on continent wide levels.
Joshua 1:9 "Be strong and of a good courage; be not afraid, neither be thou dismayed: for the Lord thy God is with thee"
(Score: 2) by JoeMerchant on Friday June 07 2019, @03:13PM (1 child)
No, when I say "Population control is coming", I'm looking at the trends of the last 70 years and inferring that either A) we're going to take a technological step backwards and make people all get out and grow their own food again, B) we're going to collapse the global ecosystem to a point that it just can't support the human population anymore, or we're going to do more to limit population growth than C) just putting our fertile 15-50 year olds to school and work, burden them with debt and entertain them so well that they are too self centered to want a bunch of children.
If you put plan C) in action throughout the world (Africa, etc.) you accelerate B). After B) you get A) by default.
All the "Malthus was wrong and his ideas will always be wrong" feel good, population isn't growing in the places that are "doing well" by economic standards still ignores the problem that growth, globally, isn't reversing, or stopping - it's barely slowing and continues to climb at a completely unsustainable rate.
🌻🌻🌻 [google.com]
(Score: 1) by khallow on Friday June 07 2019, @09:04PM
Links to your likely disagreement are already present in the above link from the last time.
Population growth from birth rate has already reversed in the developed world which is a large part of the world (and gaining new members every decade). It's only through immigration of high fertility foreigners (and their immediate descendants) that most of the place has positive population growth. Similarly, the ecological harm thing has been mostly solved as well. And of course, the whole category has tremendous personal wealth.
Then we get to the rest of world and see this remarkable, near universal shift of the entire world towards the developed world in terms of population growth and wealth.
The bottom line? Malthus indeed got it wrong. Not because of technology or our ability to make more useful things out of less resources (including food). But because he didn't know what would happen to human fertility when women were empowered.
(Score: 2) by JoeMerchant on Friday June 07 2019, @03:41PM (5 children)
This isn't an us vs. them problem. We all breathe the same air, fish the same oceans. If the problem isn't addressed head-on, if global population growth is allowed to continue on the present curve for the next 50 years, that will be implicit mass-murder on a planet wide scale.
Meanwhile, let's train up a bunch of CO2 generating AIs so we have more time for CO2 generating recreation - as long as that recreation doesn't include procreation, it might work out o.k.
🌻🌻🌻 [google.com]
(Score: 2) by Osamabobama on Friday June 07 2019, @07:07PM (4 children)
You mean this curve? [ourworldindata.org] Doesn't look so bad...
Here's the context. [ourworldindata.org]
Appended to the end of comments you post. Max: 120 chars.
(Score: 2) by JoeMerchant on Friday June 07 2019, @07:21PM (3 children)
I've always considered those quite deceptive, A) because the happy news is pretty much all on the projection side, and B) they reassure everyone that the problem has dropped to near zero at some point 50-100 years in the future, a future so unknowably remote from today that any projections are completely meaningless.
Take your graph and mask out the crystal ball gazing section, then tell me how bad it does or doesn't look.
Consider, also, that we've already f-ed the oceans at a human population of 7.5B, but this "not so bad" projection is anticipating another 30% population growth, and an ever growing percentage of the world population expects to be driving cars, flying around on jets, and living in heated & cooled houses with food delivered from around the globe to the corner store.
🌻🌻🌻 [google.com]
(Score: 1) by khallow on Friday June 07 2019, @09:15PM (2 children)
Those graphs show a substantial, global drop in population growth over the past 50 years.
Not badly, of course. One also has to consider the actual harm caused, not merely the size of the object which the harm has occurred.
(Score: 2) by JoeMerchant on Saturday June 08 2019, @02:08AM (1 child)
No, those graphs show a substantial, global drop in the rate of increase of population growth. Population is still growing, and growing at a substantial rate, perhaps not the total insanity of the past couple of decades, but insanity nonetheless, with no real end in sight.
Not badly, just the 6th mass extinction event in the history of life on the planet - who cares about biodiversity anyway, right? More room to grow blonde haired blue eyed Aryans, wheat and cows, those are the best things nature had to offer, we should focus on perfecting those three genomes and whatever we need to serve them. What else do we need, really? Oh, yeah, hops and yeast [npr.org] would be nice, too. Just like the Germans of 1516 didn't know that they needed yeast to make beer, there's more to the Earth's ecosystem that we don't know than that which we have figured out so far, and we're destroying it much faster than we're understanding it.
🌻🌻🌻 [google.com]
(Score: 1) by khallow on Saturday June 08 2019, @07:03AM
Indeed.
Except within 50 to 100 years when the global rate of increase goes below zero.
Most of that mass extinction happened well before human civilization was even a thing (for example, the last of the mammoths died around 6000 BC). As to caring, the developed world cares and has gone to considerable lengths to restore regional biodiversity. Meanwhile, the poor parts of the world can't afford to care. A trend to think about.
It's interesting how stilted your attempts at sarcasm are. I wonder if a bunch of German Nazis from the late 1930s could make that any more stilted? Throw in something about the Jews, the extended bouts of forced laughter, and the obligatory praise for the Fuhrer could make this more painful an experience. Maybe.
"We" being Asia and Africa at present. Funny how blame diffusion works. When everyone is to blame, then nobody is to blame.
But I'm still left to wonder. What was the point of your post? The first part was a rather trivial error on my part (I was speaking of growth rate). The second part is completely irrelevant to any discussion of the story. Nobody is doing that.
What is really annoying here is that you've been corrected before and yet you come back with this same nonsense. It's like nothing stuck.
(Score: 0, Troll) by khallow on Friday June 07 2019, @07:13PM
Now we just have to force reality to come up with a relevant question.
Ignore that the developed world already figured out the problems that you claim to be concerned about.
Rule out other options for getting what is wanted here, such as jobs in call centers. Don't need UBI with a good paying job. We'll "train an AI" to get rid of those pesky alternatives.
(Score: 0) by Anonymous Coward on Friday June 07 2019, @03:46PM (1 child)
> But, once trained, what's the value of this AI? Can it reduce staff in a call center?
From reading TFA, I got the impression that the value of most of these expensive AIs are one PhD thesis. And if it works well, maybe a few good papers?
(Score: 3, Informative) by JoeMerchant on Saturday June 08 2019, @01:58AM
It takes a lot of PhD thesi before you get one good practical application, but, like venture capital, you never know which one out of the group spawns a unicorn, and which 19 are losers.
🌻🌻🌻 [google.com]
(Score: 4, Funny) by inertnet on Friday June 07 2019, @12:15PM (2 children)
Why does this AI need 5 cars?
(Score: 4, Funny) by takyon on Friday June 07 2019, @12:28PM (1 child)
To crash them into nearby humans simultaneously.
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 4, Funny) by Osamabobama on Friday June 07 2019, @06:59PM
That's solving the trolley problem [wikipedia.org] with out-of-the-box thinking.
Appended to the end of comments you post. Max: 120 chars.
(Score: 2) by takyon on Friday June 07 2019, @12:27PM
Switch to neuromorphic or a future CPU/GPU/TPU, and power consumption will plummet.
Your trained algorithm can also be copied endlessly, unlike oil. Of course, copying data also emits CO2. Don't copy that floppy!
[SIG] 10/28/2017: Soylent Upgrade v14 [soylentnews.org]
(Score: 4, Interesting) by JoeMerchant on Friday June 07 2019, @01:41PM (2 children)
Above, I said I'd much rather add value to society doing just about anything OTHER than what an AI can be trained to do. Now, I realize that there are actually happy call center workers out there, and I can imagine some circumstances where a core group of "superstar" call center workers could be highly compensated and used to help train AI to do the bulk of their work.
That, in a nutshell, is what I think the future of AI oriented jobs are. Instead of an "on demand" meat based labor market where millions of people are hired en-masse for low pay, quickly trained, used, and then laid off when demand slacks off - imagine a core group of maybe 6 call center experts who handle 2-3 hours of calls per day and spend the rest of their time reviewing each other's performance and training AI to "do it right," in a sort of closed loop continuous improvement process. Then, the AI can do the work of hundreds when needed, and spin down - emit less CO2 - when not needed. The core group of human call center workers could rotate out and get PhDs in psychology, marketing, advanced basket weaving, whatever helps them to be better call center workers, and they could be paid well because there are only a few of them and they are not expected to burn out and quit at any moment.
The same thing applies for doctors, lawyers, judges, etc. Reduce current meat based staffing levels and augment with AI to provide higher levels of service. Treat the meat better than current standards, get better outcomes, and save money in the process. What to do with all this saved money? Start with UBI for the displaced workers and let them pursue their dreams, whether those be entrepreneurship, volunteerism, the arts, or navel contemplation, society is better off with lots of people living frugally on UBI than the same people stressed out and struggling to make ends meet.
Oh, just don't let the UBI crowd go out and make lots of babies - that's something that's got to stop no matter what, but it could get out of hand even faster with UBI.
🌻🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Friday June 07 2019, @04:02PM (1 child)
Have you read:
https://theartofresearch.org/ai-ubi-and-data/ [theartofresearch.org]
Vi Hart was originally taken by the seemingly virtuous pairing of AI and UBI, but then she did a deep dive and isn't so sure anymore... A few snippets, the link is long but worth a read imo,
(Score: 0) by Anonymous Coward on Friday June 07 2019, @06:08PM
Another snippet,
(Score: 2) by Rupert Pupnick on Friday June 07 2019, @10:13PM
Like many commenters, I agree that 5 cars worth of carbon footprint doesn’t seem a big price to pay for a single AI expert system to be created and then distributed across a large number of platforms— these days potentially in the millions. By singling out that one energy gobbling outlier (trial and error algorithm? I guess that’s evolution) the article is showing that if the energy costs of training aren’t managed, things could get ugly. It’s a good awareness raising piece, and doesn’t strike me as sensational, alarmist, or phony at all.