Over the past two years, the US stock market's AI industry-related sectors have collectively created trillions of dollars in market value. However, the newly added market volume is extremely unevenly distributed: NVIDIA alone has a market value of $4.5 trillion with a 73% profit margin; when the annual revenue of OpenAI and Anthropic is combined, it only amounts to $450 billion; downstream companies like CoreWeave, Cursor, and Perplexity are raising funds while burning cash. The AI industry exhibits a triangular value structure where the further upstream, the higher the profit margin.
Where is the money flowing, and where will it flow in 10 years? This is the core question that the newly launched MS&E 435 course at Stanford this semester is attempting to dissect. Host Apoorv Agrawal, from the investment firm Altimeter, has invited nine key industry players to try to piece together the future flow of AI industry value from the perspective of industry leaders.
Triangular Value Distribution
In early 2024, Agrawal published a report titled "The Economics of Generative AI," concluding that the chip layer consumes 83% of the industry's profit.
Two years later, the ecosystem's total volume has expanded from $900 billion to $4.35 trillion. The profit share of the chip layer still dominates, dropping from 83% to 79%.
Breaking it down, the chip layer has an annual revenue of about $3 trillion, with NVIDIA taking home 80%; the infrastructure layer about $750 billion; and the application layer about $600 billion. The profit margins for the three layers are 73%, 55%, and 33%, respectively. In terms of absolute profit, they are $2.25 trillion, $400 billion, and $200 billion.
The profit profile of the AI industry, in stark contrast to the cloud computing industry that drove the previous tech industry growth, shows that in the traditional cloud computing stack, the chip layer only takes 6% of the profit, while the application layer takes 70%.
Agrawal astutely summarized the current competitive landscape: the chip layer is a single-player game, the application layer is a two-player game, and the intermediate infrastructure layer is the only true multiplayer battlefield.
NVIDIA's Solo Performance
The second session was co-hosted by Altimeter's partner Brad Gerstner and NVIDIA's Sunny Madra. Sunny, originally an investor in Groq, was involved in facilitating the acquisition of Groq by NVIDIA. These two explained why NVIDIA will continue to dominate alone in the chip layer.
Brad offered a counterintuitive valuation: NVIDIA with a $4.5 trillion market cap, a P/E ratio of 13x, only halfway to the market average, and a 70% annual revenue growth. He publicly predicted that NVIDIA would become the world's first $10 trillion company. His reasoning was that over the next eight quarters, NVIDIA had already secured trillion-dollar orders, with demand far exceeding supply. Huang Renxun once told Brad that he believed demand would increase by a factor of 1 billion.
Behind this confidence lies the technical logic of transitioning from pre-training to the era of inference—where the computational cost to generate each token is the model's parameter count multiplied by the square of the context length.
There is no shortage of challengers in the realm of custom chips. Google's 7th-gen TPU Ironwood has entered mass production, with Anthropic placing an order for 1 million units, rumored to have directly pushed NVIDIA to reduce prices by 30% for some customers; Amazon's Trainium2 chip has been deployed in 1.4 million instances, generating over $10 billion in annualized revenue; Microsoft's Maia 200 went live on Azure in January, and OpenAI also signed a contract with Broadcom for a 10-gigawatt custom ASIC.
Huang Renxun casually remarked, "Many ASIC projects ultimately get scrapped." Historical evidence supports his statement. Furthermore, even if TPUs, Trainium, and Maia all prove successful, NVIDIA will still maintain its lead. The examples mentioned do not indicate that NVIDIA is irreplaceable but rather underscore the sheer size of this market.
Expensive Electrician
The keynote speaker for the third session was Chase Lochmiller, the founder of Crusoe. In Abilene, a small town in western Texas, Crusoe built a 2.1-gigawatt data center campus, which is the largest private substation in the U.S., consuming electricity equivalent to two Denver cities. The initial companies to move in were Oracle and OpenAI. The campus employs 9,000 construction workers year-round, a significant number considering the town's population is only 120,000.
In a slide, Chase broke down the cost breakdown of each megawatt of power: with a total cost of around $19 million, the largest single item is labor, reaching up to $4.7 million per megawatt, with chip and cooling equipment costs trailing far behind. By this calculation, for a gigawatt-scale campus, just the labor cost alone would burn $4.7 billion annually.
In addition to labor costs, the cost per megawatt of a gas turbine has increased from $1 million to $3 million within three years. The reason behind this is that the four major manufacturers — GE Vernova, Siemens, Mitsubishi Heavy Industries, and Pratt & Whitney — have maintained their production capacity, while demand has multiplied several times over.
Chase is bearish on legacy electrical equipment giants like Eaton and Schneider, which have remained relatively unchanged for over a century. In the long run, the entire power architecture will need to be redesigned from 765 kV high voltage to 900 V DC inside cabinets. However, in the short term, these established factories will continue to benefit.
Organizational Structure Lagging Behind AI Development
The guest for the fourth session was Ali Ghodsi, CEO of the software company Databricks. He started off with a groundbreaking statement: AGI has already been achieved. According to him, current AI models have long met the 2009 Berkeley AMP Lab's definition of AGI. The goal has already been scored; it's just that people have been pulling the goalposts back.
The reason people believe AI has not yet met expectations is actually due to themselves.
A report from MIT shows that 95% of companies' AI pilots have failed. Ali explains this by stating that the models lack the unwritten context that exists within organizations.
Every company has that one employee who has worked for twenty years, and everyone goes to him when they encounter a problem. However, the knowledge in his mind has never been captured in any model.
As a comparison, the electric motor was invented in 1880 but did not significantly increase productivity in statistical terms until 1920. During those forty years, companies simply replaced steam engines with electric motors without considering that the entire factory layout should be completely redesigned.
Databricks learned from its own mistakes. A data connector that originally would have taken three quarters to deliver was first handed over to AI, saving only one and a half months. Later, when a person who was willing to scrap the entire process and start over took charge, the entire project was delivered in one quarter. The real time saver was not the upgraded model version but the person who dismantled and rebuilt the process.
Ali believes that a significant opportunity at the application layer will eventually emerge and will be concentrated in the hands of players who are willing to rewrite organizational logic. The speed of this advancement depends on people rather than on GPT-6 or Opus-5.
Inverted Triangle
In the technology stack, value naturally flows from the underlying hardware to the upper layers of software and applications.
The cloud computing industry has completed a 15-year journey from hardware-centric to software-centric. The AI stack needs to undergo a similar value inversion. It either needs to continue to thrive at the application layer, or wait for the chip layer's gross margin to converge from its high of 73% to a hardware-like 6% in the cloud era. Currently, both transitions are happening, but neither at a sufficient pace. Based on the pace of transformation in the past two years, to catch up with the profit share at the level of the cloud computing platform in its heyday, the application layer will need at least a ten-year head start.
Therefore, betting on the chip layer at the moment is a bet on cash flow that can be realized in the next two years; betting on the application layer is a bet on the major trend of value appreciation in the next five to ten years.
Before the technological revolution that will lead to the collapse of chip layer gross margins arrives, the closer you are to the chip, the closer you are to the profit.
Welcome to join the official BlockBeats community:
Telegram Subscription Group: https://t.me/theblockbeats
Telegram Discussion Group: https://t.me/BlockBeats_App
Official Twitter Account: https://twitter.com/BlockBeatsAsia
