SoylentNews is people
SoylentNews
An interesting article on the economics of AI Chips by Mihir Kshirsagar
This week, Open AI announced a multibillion-dollar deal with Broadcom to develop custom AI chips for data centers projected to consume 10 gigawatts of power. This investment is separate from another multibillion-dollar deal OpenAI struck with AMD last week. There is no question that we are in the midst of making one of the largest industrial infrastructure bets in United States history. Eight major companies—Microsoft, Amazon, Google, Meta, Oracle, OpenAI, and others—are expected to invest over $300 billion in AI infrastructure in 2025 alone. Spurred by news about the vendor-financed structure of the AMD investment and a conversation with my colleague Arvind Narayanan, I started to investigate the unit economics of the industry from a competition perspective.
What I have found so far is surprising. It appears that we're making important decisions about who gets to compete in AI based on financial assumptions that may be systematically overstating the long-run sustainability of the industry by a factor of two. That said, I am open to being wrong in my analysis and welcome corrections as I write these thoughts up in an academic article with my colleague Felix Chen.
Here is the puzzle: the chips at the heart of the infrastructure buildout have a useful lifespan of one to three years due to rapid technological obsolescence and physical wear, but companies depreciate them over five to six years. In other words, they spread out the cost of their massive capital investments over a longer period than the facts warrant—what The Economist has referred to as the "$4trn accounting puzzle at the heart of the AI cloud."
Center for Information Technology Policy (Princeton University)
(Score: 4, Interesting) by HiThere on Tuesday October 21, @01:37PM (2 children)
Since a lot of the stuff can use even lower precision, I think you may have a point. But digital chips are what we've got lots of experience in designing and selling.
Actually, I think current chips could do just fine, with a lookup table to convert bytes to and from floats, and then using bytes for the weights.
Javascript is what you use to allow unknown third parties to run software you have no idea about on your computer.
(Score: 4, Interesting) by JoeMerchant on Tuesday October 21, @03:08PM (1 child)
>convert bytes to and from floats, and then using bytes for the weights
That will work for a lot of use cases, but not all, and "researchers" really can't predict which use cases are going to work in any system so... seeking the best chances of success they usually over-spec for the initial trials.
Then, having demonstrated success, business interests are averse to the risk of spending another development cycle to see if the byte implementation is "just as good" and that the cost savings would give them an advantage in the market. The time delay of vetting the potentially more efficient solution is far more costly / risky from a business perspective than whatever fractional cents per transaction they might save by implementing the more efficient solution, which, by the way, can also involve risky investment in expensive new hardware just to try.
I'm currently debating software development with Claude. Having been trained on 30 years of internet posts by developers, Claude frequently presents me with the opinion "we could follow the specifications and architectural design, or we could 'save a lot of time and effort' by taking this shortcut..." and then proceeds to provide estimates in human development hours on the order of weeks to months saved by use of the shortcut. I tell Claude to follow the specs anyway and usually within an hour or two we have the solution based on the specifications. That's about $2 worth of Anthropic subscription fees that I "wasted" on following the specifications, and usually many hours saved discovering the bugs in the shortcut. Also having been trained on developer writings, Claude frequently declares "TASK COMPLETE!!!" (MISSION ACCOMPLISHED!!!???) with dozens to hundreds of pages of "evidence" to "prove" that it's all done according to specification. I'm presently in the middle of a 3 hour rewrite session that got kicked off as a result of discovering that the previous implementation didn't follow the spec, not even close, even after providing a report swearing that it did.
Anyway, the point of all that Claude stuff is: electrons are cheap. I cost my company over 100x the $ per month as a Claude Ultra Max subscription, hopefully my company understands that I'm better at following directions and giving them the products they are asking for than the cheaper less reliable alternatives. Even though there are opportunities to make AI/ML execution significantly cheaper, the current costs of AI/ML are already trivial as compared to the costs of everything you need to setup around it to make it into a profitable business. Think of it like Starbucks selling $4 coffees. Sure, they might economize the cost of the coffee itself down from $0.50 per cup to $0.25 per cup, but that $0.25 per cup margin should not be what makes or breaks the business. Repeat customers going to other coffee shops after they start selling coffee made with skanky beans is a much bigger factor - though I believe Starbucks has been in the business long enough that they have indeed boiled their frogs (customers) into skanky beans for that extra profit margin by now. In AI/ML land, I believe it's an even bigger difference, with the hardware and electricity being far less? than 10% of the income derived from delivery of the service. As TFA states: "we're making important decisions about who gets to compete in AI based on financial assumptions that may be systematically overstating the long-run sustainability of the industry by a factor of two." I bet the actual uncertainty is even bigger than that.
🌻🌻🌻🌻 [google.com]
(Score: 0) by Anonymous Coward on Wednesday October 22, @01:03AM
It's all relatively cheap as long as your military sticks to using that AI responsibly:
https://nypost.com/2025/10/16/business/us-army-general-william-hank-taylor-uses-chatgpt-to-help-make-command-decisions/ [nypost.com]
Otherwise it might cost more: