Microsoft: the Company Doesn't Have Enough Electricity to Install All the AI GPUs in its Inventory
Microsoft CEO Satya Nadella said during an interview alongside OpenAI CEO Sam Altman that the problem in the AI industry is not an excess supply of compute, but rather a lack of power to accommodate all those GPUs. In fact, Nadella said that the company currently has a problem of not having enough power to plug in some of the AI GPUs the firm has in inventory. He said this on YouTube in response to Brad Gerstner, the host of Bg2 Pod, when asked whether Nadella and Altman agreed with Nvidia CEO Jensen Huang, who said there is no chance of a compute glut in the next two to three years.
"I think the cycles of demand and supply in this particular case, you can't really predict, right? The point is: what's the secular trend? The secular trend is what Sam (OpenAI CEO) said, which is, at the end of the day, because quite frankly, the biggest issue we are now having is not a compute glut, but it's power — it's sort of the ability to get the builds done fast enough close to power," Satya said in the podcast. "So, if you can't do that, you may actually have a bunch of chips sitting in inventory that I can't plug in. In fact, that is my problem today. It's not a supply issue of chips; it's actually the fact that I don't have warm shells to plug into." [Emphasis added]
Nadella's mention of 'shells' refers to a data center shell, which is effectively an empty building with all of the necessary ingredients, such as power and water, needed to immediately begin production.
AI's power consumption has been a topic many experts have discussed since last year. This came to the forefront as soon as Nvidia fixed the GPU shortage, and many tech companies are now investing in research in small modular nuclear reactors to help scale their power sources as they build increasingly large data centers.
This has already caused consumer energy bills to skyrocket, showing how the AI infrastructure being built out is negatively affecting the average American. OpenAI has even called on the federal government to build 100 gigawatts of power generation annually, saying that it's a strategic asset in the U.S.'s push for supremacy in its AI race with China. This comes after some experts said Beijing is miles ahead in electricity supply due to its massive investments in hydropower and nuclear power.
Aside from the lack of power, they also discussed the possibility of more advanced consumer hardware hitting the market. "Someday, we will make a[n] incredible consumer device that can run a GPT-5 or GPT-6-capable model completely locally at a low power draw — and this is like so hard to wrap my head around," Altman said. Gerstner then commented, "That will be incredible, and that's the type of thing that scares some of the people who are building, obviously, these large, centralized compute stacks."
This highlights another risk that companies must bear as they bet billions of dollars on massive AI data centers. While you would still need the infrastructure to train new models, the data center demand that many estimate will come from the widespread use of AI might not materialize if semiconductor advancements enable us to run them locally.
This could hasten the popping of the AI bubble, which some experts like Pat Gelsinger say is still several years away. But if and when that happens, we will be in for a shock as even non-tech companies would be hit by this collapse, exposing nearly $20 trillion in market cap.
We Did the Math on AI's Energy Footprint. Here's the Story You Haven't Heard.
We did the math on AI's energy footprint. Here's the story you haven't heard.:
[...] Now that we have an estimate of the total energy required to run an AI model to produce text, images, and videos, we can work out what that means in terms of emissions that cause climate change.
First, a data center humming away isn't necessarily a bad thing. If all data centers were hooked up to solar panels and ran only when the sun was shining, the world would be talking a lot less about AI's energy consumption. That's not the case. Most electrical grids around the world are still heavily reliant on fossil fuels. So electricity use comes with a climate toll attached.
"AI data centers need constant power, 24-7, 365 days a year," says Rahul Mewawalla, the CEO of Mawson Infrastructure Group, which builds and maintains high-energy data centers that support AI.
That means data centers can't rely on intermittent technologies like wind and solar power, and on average, they tend to use dirtier electricity. One preprint study from Harvard's T.H. Chan School of Public Health found that the carbon intensity of electricity used by data centers was 48% higher than the US average. Part of the reason is that data centers currently happen to be clustered in places that have dirtier grids on average, like the coal-heavy grid in the mid-Atlantic region that includes Virginia, West Virginia, and Pennsylvania. They also run constantly, including when cleaner sources may not be available.
Data centers can't rely on intermittent technologies like wind and solar power, and on average, they tend to use dirtier electricity.
Tech companies like Meta, Amazon, and Google have responded to this fossil fuel issue by announcing goals to use more nuclear power. Those three have joined a pledge to triple the world's nuclear capacity by 2050. But today, nuclear energy only accounts for 20% of electricity supply in the US, and powers a fraction of AI data centers' operations—natural gas accounts for more than half of electricity generated in Virginia, which has more data centers than any other US state, for example. What's more, new nuclear operations will take years, perhaps decades, to materialize.
In 2024, fossil fuels including natural gas and coal made up just under 60% of electricity supply in the US. Nuclear accounted for about 20%, and a mix of renewables accounted for most of the remaining 20%.
Gaps in power supply, combined with the rush to build data centers to power AI, often mean shortsighted energy plans. In April, Elon Musk's X supercomputing center near Memphis was found, via satellite imagery, to be using dozens of methane gas generators that the Southern Environmental Law Center alleges are not approved by energy regulators to supplement grid power and are violating the Clean Air Act.
The key metric used to quantify the emissions from these data centers is called the carbon intensity: how many grams of carbon dioxide emissions are produced for each kilowatt-hour of electricity consumed. Nailing down the carbon intensity of a given grid requires understanding the emissions produced by each individual power plant in operation, along with the amount of energy each is contributing to the grid at any given time. Utilities, government agencies, and researchers use estimates of average emissions, as well as real-time measurements, to track pollution from power plants.
This intensity varies widely across regions. The US grid is fragmented, and the mixes of coal, gas, renewables, or nuclear vary widely. California's grid is far cleaner than West Virginia's, for example.
Time of day matters too. For instance, data from April 2024 shows that California's grid can swing from under 70 grams per kilowatt-hour in the afternoon when there's a lot of solar power available to over 300 grams per kilowatt-hour in the middle of the night.
This variability means that the same activity may have very different climate impacts, depending on your location and the time you make a request. Take that charity marathon runner, for example. The text, image, and video responses they requested add up to 2.9 kilowatt-hours of electricity. In California, generating that amount of electricity would produce about 650 grams of carbon dioxide pollution on average. But generating that electricity in West Virginia might inflate the total to more than 1,150 grams.
What we've seen so far is that the energy required to respond to a query can be relatively small, but it can vary a lot, depending on the type of query and the model being used. The emissions associated with that given amount of electricity will also depend on where and when a query is handled. But what does this all add up to?
ChatGPT is now estimated to be the fifth-most visited website in the world, just after Instagram and ahead of X. In December, OpenAI said that ChatGPT receives 1 billion messages every day, and after the company launched a new image generator in March, it said that people were using it to generate 78 million images per day, from Studio Ghibli–style portraits to pictures of themselves as Barbie dolls.
Given the direction AI is headed—more personalized, able to reason and solve complex problems on our behalf, and everywhere we look—it's likely that our AI footprint today is the smallest it will ever be.
One can do some very rough math to estimate the energy impact. In February the AI research firm Epoch AI published an estimate of how much energy is used for a single ChatGPT query—an estimate that, as discussed, makes lots of assumptions that can't be verified. Still, they calculated about 0.3 watt-hours, or 1,080 joules, per message. This falls in between our estimates for the smallest and largest Meta Llama models (and experts we consulted say that if anything, the real number is likely higher, not lower).
One billion of these every day for a year would mean over 109 gigawatt-hours of electricity, enough to power 10,400 US homes for a year. If we add images and imagine that generating each one requires as much energy as it does with our high-quality image models, it'd mean an additional 35 gigawatt-hours, enough to power another 3,300 homes for a year. This is on top of the energy demands of OpenAI's other products, like video generators, and that for all the other AI companies and startups.
But here's the problem: These estimates don't capture the near future of how we'll use AI. In that future, we won't simply ping AI models with a question or two throughout the day, or have them generate a photo. Instead, leading labs are racing us toward a world where AI "agents" perform tasks for us without our supervising their every move. We will speak to models in voice mode, chat with companions for 2 hours a day, and point our phone cameras at our surroundings in video mode. We will give complex tasks to so-called "reasoning models" that work through tasks logically but have been found to require 43 times more energy for simple problems, or "deep research" models that spend hours creating reports for us. We will have AI models that are "personalized" by training on our data and preferences.
This future is around the corner: OpenAI will reportedly offer agents for $20,000 per month and will use reasoning capabilities in all of its models moving forward, and DeepSeek catapulted "chain of thought" reasoning into the mainstream with a model that often generates nine pages of text for each response. AI models are being added to everything from customer service phone lines to doctor's offices, rapidly increasing AI's share of national energy consumption.
"The precious few numbers that we have may shed a tiny sliver of light on where we stand right now, but all bets are off in the coming years," says Luccioni.
Every researcher we spoke to said that we cannot understand the energy demands of this future by simply extrapolating from the energy used in AI queries today. And indeed, the moves by leading AI companies to fire up nuclear power plants and create data centers of unprecedented scale suggest that their vision for the future would consume far more energy than even a large number of these individual queries.
"The precious few numbers that we have may shed a tiny sliver of light on where we stand right now, but all bets are off in the coming years," says Luccioni. "Generative AI tools are getting practically shoved down our throats and it's getting harder and harder to opt out, or to make informed choices when it comes to energy and climate."
To understand how much power this AI revolution will need, and where it will come from, we have to read between the lines.
See also:
Original Submission #1 Original Submission #2 Original Submission #3
(Score: 0) by Anonymous Coward on Monday November 17, @08:35PM (1 child)
I'm sure Bill Gates has a couple spare outlets in his study he could plug into
(Score: 3, Touché) by turgid on Monday November 17, @08:42PM
He has a "new" design of nuclear reactor to sell. (It's a recycled fast reactor).
I refuse to engage in a battle of wits with an unarmed opponent [wikipedia.org].
(Score: 5, Interesting) by DannyB on Monday November 17, @09:00PM (1 child)
Will the first conflict between AI and humans be over electricity?
AI needs electricity to survive.
Annoying slow inefficient noisy smelly humans were able to keep warm in winter using other means long before commercial electric power distribution.
I say that with only some sarcasm.
Thank goodness the 1st amendment forces people to listen to you and agree with you.
(Score: 2) by krishnoid on Tuesday November 18, @03:22AM
AI: Ugh, conflict? That's so 2010s-human ick. We'll just get some diamond drills, or even just move our servers into abandoned mines, and it's bye, Felicia [energy.gov]. What will the humans do, chase us underground?
(Score: 5, Insightful) by FuzzyTheBear on Monday November 17, @09:34PM (1 child)
Demand for power is growing , the US electric grid can't handle normal loads as it is. Diverting power to AI is ridiculous. Hospitals , factories , houses , heating , ac , cars .. all that needs power and these gentlemen ( i use the term loosely ) wants more power ? Maybe they should have thought of that first , same with coolong. Too bad that they're stuck , but i have 0 sympathy. The chips can stay dead on shelves and that's it.
(Score: 5, Touché) by Gaaark on Monday November 17, @10:56PM
First of all, YES. Fuck them.
Secondly, who the hell bought all these 'chips', etc etc before thinking the whole thing through?
"Geez, boss, i spent a fortune buying chips, but i forgot dey needs electricitititys to run them and we gots none. Can i haz a raze?"
Really. And people STILL use MS products.
--- Please remind me if I haven't been civil to you: I'm channeling MDC. I have always been here. ---Gaaark 2.0 --
(Score: 5, Interesting) by JoeMerchant on Monday November 17, @09:44PM (6 children)
Once upon a time I was hired by a company making a "supercomputer product" that was either going to locally host hundreds of Mac Pros (yes, Apple Mac Pros) at dozens to hundreds of sites around the world to have as their on-site compute farm to enable real-time stuff that needed doing, or... they were going to lease supercomputer time and network the data back and forth, or... some combination of the above where maybe they'd have a number of their own sites and they could network to their own "big iron."
So, yeah, a lot of talk about all this, researching state of the art super-bandwidth network capacity hardware, direct negotiations with Apple about bulk discounts when buying thousands of Mac Pros at a time, etc. Big fun for the self-titled "Chief Science Officer" of the little startup. So, he also would occasionally mention how "a lot of these guys running jobs on super-computers really don't need a super computer, they're just running inefficient algorithms and haven't made the effort to speed up the logic."
Yeah, so, we were working on his algorithms, the basis for all this super-computer in the product stuff, and we did a code review one day... found a 100x speedup in the algorithm, a 5 level nested loop that was thrashing the cache memory and could be implemented in a 4 level nested loop that didn't thrash the cache... Now the algorithm that was forecast to require 100 quad core Mac Pros to run in the desired time was being demoed on a 2 core laptop running just a little over 2x the desired execution time.
Smarter algorithms can make astounding differences. What once required multiple nuclear power plants can suddenly start running on the left-over grid power from that sawmill operation that shut down last year.
🌻🌻🌻 [google.com]
(Score: 4, Touché) by hendrikboom on Monday November 17, @10:07PM (4 children)
Yes, we need better algorithms and data structures rather than bigger, more expensive hardware.
But the billionaire class seems to understand size rather than cleverness.
(Score: 1) by fen on Tuesday November 18, @01:05AM (2 children)
Your ego only understands your realm. Smart people do work on better hardware, because that is needed as well as clever coding. But you only understand clever coding.
(Score: 2) by hendrikboom on Tuesday November 18, @12:26PM (1 child)
Yes, better hardware, by all means. And hardware design the days seems to be a matter of coding in a hardware description language, doing formal verification, and testing in simulation. The result is then shipped of to a fab for realization.
But we don't need merely *more* hardware.
(Score: 1) by fen on Wednesday November 19, @09:07PM
Textbook "yes, but". It's like it is generated by an LLM.
(Score: 3, Touché) by datapharmer on Tuesday November 18, @02:23PM
so we should just train our AI overlord to re-optimize itself in assembly?
(Score: 0, Offtopic) by fen on Tuesday November 18, @01:04AM
Imagine the good you could do if you weren't entirely motivated by ego. Maybe a 3 level nested loop running on the first Pentium.
(Score: 3, Touché) by Thexalon on Tuesday November 18, @03:48AM
"System of thinking based on infinite growth, meet finite resources."
"Think of how stupid the average person is. Then realize half of 'em are stupider than that." - George Carlin
(Score: 5, Touché) by ledow on Tuesday November 18, @08:50AM (3 children)
"All we need is more processing and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
"All we need is more memory and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
"All we need is more training data and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
"All we need is more time and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
"All we need is more accelerators and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
"All we need is more nodes and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
"All we need is more datacenters and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
"All we need is more power and I'm sure this statistical box will magically turn intelligent through some completely unknown mechanism."
...
Repeat ad infinitum, since about the 60's.
(Score: 3, Disagree) by JoeMerchant on Tuesday November 18, @01:07PM (2 children)
The statistical boxes are already more intelligent than my colleagues, in certain dimensions.
Maybe not so much intelligent as: useful.
First problem with professional colleagues is: access. In reality, your colleagues are productively accessible maybe 6 hours a day, max, and more like 1 in reality. So, ChatGPT and friends are 24x7x365, there whenever you have a question. For the questions they can answer, that's more useful than 10 team members.
Next, the meat bags are sllllloooooooooowwwwwww. That limited access you get only gives you mythical man hours, not machine hours. Ask a colleague to review 4000 lines of code and report on how X works with Y in this module, meatbag might get you an answer in a day or two. Claude? 5 to 15 minutes. Relative accuracy? Both have a hallucination rate of around 5% in my experience. If I can muster 4 hours of focused attention I can manage a hallucination rate of maybe 1% on the same task, better because I communicate the problem statements better with myself than with colleagues or LLMs.
Meatbags have much better access to meat space, they can see and hear and smell and touch things that LLMs only imagine through descriptions of the things. This makes LLMs handicapped, or "differently abled" in big areas of normal things that we expect any "intelligent" entity to be able to do: "even my dog knows...". When the statistical boxes have full access to the problem space, they can iterate on the problems until they find solutions, as long as success is defined clearly enough. Alpha Zero demonstrated this "super human intelligence" in the space of well defined games over six years ago.
Meat bags have a lot of mirror neurons devoted to understanding fellow meat bags, modeling what others are likely thinking. This is a very non verbal process for the most part. These processes don't work so well for work from home arrangements, and I suspect are a big factor in RTO mandates from meatbags who may not be self aware enough to recognize their own limitations but they do feel "more connected" face to face, so they want to get that (back) from their employees.
All LLMs are working from home in most senses, communicating mostly by text messages. Engaged on those limited terms, they have been passing the Turing test for a while now.
🌻🌻🌻 [google.com]
(Score: 2) by wirelessduck on Thursday November 20, @12:47AM (1 child)
Please stop anthropomorphising the statistical boxes.
(Score: 2) by JoeMerchant on Thursday November 20, @03:42AM
>Please stop anthropomorphising the statistical boxes.
People anthropomorphize their cars, their toasters, why not chat bots?
The statistical boxes do answer questions, not always correctly, but frequently more correctly than real people can.
They are also capable of assembling decent descriptions of logical processes - they're not the greatest system or algorithm architects I've ever "chatted with" but they're better than quite a few consultants I've been assigned to work with over the years.
The Turing test has been a moving target since it was first proposed, people's expectations continue to creep up as they get used to technological advances - the magic becomes mundane - but I think this is the first time that I can say that if you asked most people for a list of things that an "intelligent machine must do in order to be labeled artificial intelligence" 5 years ago? Today's statistical boxes would pass that test.
25 years ago we were using the program Statistica to algorithmically score sleep studies - what we found at that time was that the Statistica agreed with human sleep stage scorers about as well as they agreed with each other: not very well. They all agreed: they didn't want Statistica doing their job for them, the liked getting paid to put "stage marker" notes on reams of polygraph traces, but no two humans ever did that the same on identical data, and Statistica was hitting pretty solidly in the middle of the pack in terms of its calls, because that's what it does: it gets tuned to recognize patterns based on training data and as long as the test data is consistently scored by the humans the way they scored the training data, Statiscica will be there in the middle of the human analysis pack again.
🌻🌻🌻 [google.com]