Full text of Huang Renxun's GTC speech: trillion in revenue, LPU, space chips, one-click “shrimp farming”

Source:Wall Street News

Original title: Huang Renxun's GTC speech full text: The era of reasoning has arrived, lobster is the new operating system

On March 16, 2026, the Nvidia GTC 2026 conference officially opened. Nvidia Founder and CEO Hwang In-hoon delivered a keynote speech.

At this conference, which is regarded as the “annual pilgrimage for the AI industry,” Hwang In-hoon explained Nvidia's transformation from a “chip company” to an “AI infrastructure and factory company.” Faced with the market's biggest concern about performance sustainability and growth space, Huang Renxun detailed the underlying business logic that drives future growth — “Token Factory Economics.”

Performance guidance is extremely optimistic, “at least $1 trillion in demand in 2027”

In the past two years, global demand for AI computing has exploded exponentially. As large models evolve from “perception” and “generation” to “reasoning” and “action (execution of tasks),” the consumption of computing power has increased dramatically. In response to the order and revenue ceiling that the market is paying close attention to, Hwang In-hoon gave extremely strong expectations.

In his speech, Hwang In-hoon put it bluntly:

At this time last year, I said we were seeing $500 billion in high-confidence demand, covering Blackwell and Rubin until 2026. Right now, right here and now, I see at least $1 trillion in demand (at least $1 trillion) by 2027.

Hwang In-hoon's trillion dollar forecast once boosted Nvidia's stock price by more than 4.3%.

More than that, he added to that number:

Is that reasonable? That's what I'm going to talk about next. In fact, we'll even be in short supply. I'm sure the actual computational requirements will be much higher than that.

Hwang In-hoon pointed out that today's Nvidia system has proven itself to be the “lowest-cost infrastructure” in the world. Since Nvidia can run AI models in almost every field, this versatility allows the $1 trillion invested by customers to be fully utilized and maintained for a long life cycle.

Currently, 60% of Nvidia's business comes from the top five largest cloud service providers, while the other 40% of Nvidia's business is widely distributed in various fields such as sovereign cloud, enterprise, industry, robotics, and edge computing.

Token factory economics, performance per watt determines the lifeblood of business

To explain the rationality of this $1 trillion demand, Hwang In-hoon presented a new set of business thinking to CEOs of global companies. He pointed out that future data centers will no longer be warehouses for storing files, but “factories” that produce tokens (basic units generated by AI).

Hwang In-hoon emphasized:

Every data center and every factory is, by definition, limited by electricity. A 1GW (gigawatt) factory never becomes 2GW; this is a law of physics and atoms. At fixed power, whoever has the highest token throughput per watt has the lowest production cost.

Huang Renxun divides future AI services into the following commercial tiers:

Free tier (high throughput, low speed)

Mid-tier (~$3 per million tokens)

Premium tier (~$6 per million tokens)

High speed tier (~$45 per million tokens)

Ultra High Speed Tier (~$150 per million tokens)

He pointed out that as the model gets bigger and the context gets longer, AI will become smarter, but the token generation rate will decrease. Huang Renxun said:

In this token factory, your throughput and token generation speed will be directly converted into accurate revenue for next year.

Hwang In-hoon emphasized that Nvidia's architecture allows customers to achieve extremely high throughput in the free tier, while improving performance by an astonishing 35 times in the highest-value inference layer.

Vera Rubin achieves 350x acceleration in two years, Groq fills in extreme reasoning

Constrained by this physical limit, Nvidia introduces its most complex AI computing system ever, Vera Rubin. Huang Renxun said:

When Hopper was mentioned in the past, I would hold up a chip, that's cute. But when you think of Vera Rubin, you think of the whole system. In this 100% liquid-cooled system that completely eliminated traditional cabling, it used to take two days to install a rack, but now it only takes two hours.

Huang Renxun pointed out that through extreme end-to-end software and hardware co-design, Vera Rubin created an amazing data leap in the same 1GW data center:

In just two years, we increased the token generation rate from 22 million to 700 million, an increase of 350 times. Moore's Law can only bring about a 1.5 times improvement over the same period of time.

To address bandwidth bottlenecks under extreme inference conditions (such as 1000 Tokens/s), Nvidia proposed the final solution for integrating the acquired company Groq: asymmetric segregated reasoning. Hwang In-hoon explained:

The characteristics of these two processors are quite different. The GroQ chip has 500MB of SRAM, while a Rubin chip has 288GB of memory.

Huang Renxun pointed out that through the Dynamo software system, Nvidia handed over the “pre-fill (pre-fill)” stage requiring massive computation and video memory to Vera Rubin, and handed over the “decoding” stage, which is extremely sensitive to delays, to Groq. Huang Renxun also gave suggestions on the enterprise's computing power allocation:

If your job is mainly high throughput, 100% use Vera Rubin; if you have a large number of high-value programming-level token generation requirements, give Groq 25% of the size of the data center.

According to reports, the Groq LP30 chip, manufactured by Samsung, is already in mass production and is expected to be shipped in the third quarter, while the first Vera Rubin rack is already running on the Microsoft Azure cloud.

Furthermore, with regard to optical interconnection technology, Hwang In-hoon showcased Spectrum X, the world's first mass-produced co-packaged optical (CPO) switch, and calmed the market dispute over the “copper retreat” route:

We need more copper cable production capacity, more optical chip production capacity, and more CPO production capacity.

Agent ends traditional SaaS, and “annual salary+token” becomes standard in Silicon Valley

In addition to hardware barriers, Hwang In-hoon left a lot of space for AI software and the revolution in ecology, especially the outbreak of agents (agents).

He described the open source project OpenClaw as “the most popular open source project in human history,” saying it surpassed what Linux has achieved in the past 30 years in just a few weeks. Hwang In-hoon said bluntly that OpenClaw is essentially an “operating system” for agent computers.

Hwang In-hoon asserted:

Every SaaS (Software as a Service) company will become an AaaS (Agent-as-a-Service) company. Undoubtedly, in order to safely implement this kind of intelligence with the ability to access sensitive data and execute code, Nvidia launched an enterprise-grade NeMo Claw reference design, adding a policy engine and privacy router.

For ordinary workers, this change is also within easy reach. Hwang In-hoon described the new shape of the workplace in the future:

In the future, every engineer in our company will need an annual token budget. Their basic annual salary is probably hundreds of thousands of dollars, and I will give them about half of the amount as a token credit on top of that, so that they can achieve a 10x increase in efficiency. This is already Silicon Valley's new recruitment chip: How many tokens are in your offer?

At the end of his speech, Hwang In-hoon also “spoiled” the next-generation computing architecture Feynman, which will achieve the same level of expansion of copper wire and CPO for the first time. What is even more impressive is that Nvidia is developing the data center computer “Vera Rubin Space-1” deployed in space, completely opening up a space for imagination where AI computing power extends beyond Earth.

The full text of Huang Renxun's GTC 2026 speech, translated as follows (assisted by AI tools):

Moderator: Welcome Nvidia's founder and CEO Wong In-hoon to the stage.

Huang Renxun, Founder and CEO:

Welcome to GTC. I would like to remind everyone that this is a technical conference. I was very happy to see so many people lining up early in the morning and to see everyone present.

At GTC, we'll be focusing on three themes: technology, platforms, and ecosystems. Nvidia currently has three platforms: the CUDA-X platform, the system platform, and our newly launched AI factory platform.

Before we officially begin, I'd like to thank our warm-up hosts — Sarah Guo of Conviction, Alfred Lin of Sequoia Capital (Nvidia's first venture capitalist), and Gavin Baker, Nvidia's first major institutional investor. These three have deep insight into technology and are extremely influential across the entire technology ecosystem. Of course, I would also like to thank all the distinguished guests that I personally invited to attend today. Thanks to this all-star team.

I would also like to thank all the companies that showed up today. Nvidia is a platform company, and we have the technology, the platform, and a rich ecosystem. The companies present today represented almost all participants in the $100 trillion industry. A total of 450 companies sponsored this event, and I would like to express my deep gratitude.

With 1,000 technical forums and 2,000 speakers, the conference will cover every level of the “five-layer cake” architecture of artificial intelligence — from infrastructure such as land, power, and computer rooms, to chips, platforms, models, and all kinds of applications that ultimately drive the entire industry to take off.

CUDA: Twenty years of technical experience

This is where it all started. This year marks CUDA's 20th anniversary.

We have been working on the development of this architecture for 20 years. CUDA is a revolutionary invention — SIMT (Single Instruction Multiple Threading) technology allows developers to write programs in scalar code and extend them into multi-threaded applications, making programming far less difficult than previous SIMD architectures. We've also recently added the Tiles feature to help developers more easily program Tensor Cores (Tensor Cores) and the various mathematical computation structures that today's artificial intelligence relies on. Currently, CUDA has thousands of tools, compilers, frameworks, and libraries, hundreds of thousands of open projects in the open source community, and is deeply integrated into every technology ecosystem.

This chart reveals 100% of Nvidia's strategic logic, and I've been talking about this slide since the beginning. Among them, the most difficult to achieve and the most core element is the “installed capacity” at the bottom of the chart. Over two decades, we have accumulated hundreds of millions of GPUs and computing systems running CUDA around the world.

Our GPUs span all cloud platforms and serve almost every computer vendor and industry. CUDA's huge installed capacity is the root cause of this flywheel's continuous acceleration. Installed capacity attracts developers. Developers create new algorithms and achieve breakthroughs, and breakthroughs spawn new markets. The new market forms a new ecosystem and attracts more companies to join, thereby expanding the installed capacity — this flywheel continues to accelerate.

The number of downloads of the Nvidia library is growing at an astonishing rate. The scale is huge, and the growth rate is constantly increasing. This flywheel enables our computing platform to support massive applications and endless new breakthroughs.

More importantly, it also gave this infrastructure an extremely long useful life. The reason is obvious: there are so many applications that can run on NVIDIA CUDA, covering every stage of the AI lifecycle, various data processing platforms, and various scientific solvers. Therefore, once installed, an Nvidia GPU is extremely valuable in actual use. That's why the Ampere-architecture GPU we released six years ago is now rising in price in the cloud.

The root cause of all of this is: huge installed capacity, strong flywheels, and a broad developer ecosystem. When these factors work together, and we keep updating our software, computing costs will continue to drop. While accelerated computing greatly improves application performance, with our long-term maintenance and iteration of software, users can not only get an initial jump in performance, but also continue to enjoy the reduction in computing costs. We are willing to provide long-term support for every GPU in the world because they are fully architecturally compatible.

We're willing to do this because the installed capacity is so huge — every time a new optimization is released, millions of users can benefit. This dynamic combination enables the Nvidia architecture to continuously reduce computing costs while continuing to expand coverage and accelerate its own growth, ultimately stimulating new growth. CUDA is at the heart of it all.

From GeForce to CUDA: 25 years of evolution

And our journey with CUDA actually began 25 years ago.

GeForce — I'm sure many of you here grew up with GeForce. GeForce is Nvidia's most successful marketing project. We've been nurturing future customers since you couldn't afford the product — your parents replaced you as Nvidia's earliest users, bought our products year after year, until one day, you grew into excellent computer scientists and became real customers and developers.

This is the foundation that GeForce laid 25 years ago. Twenty-five years ago, we invented the programmable shader — an obvious yet far-reaching invention that made accelerators programmable. It was also the world's first programmable accelerator, the pixel shader. Five years later, we created CUDA — one of our most important investments ever. The company's financial resources were limited at the time, but we bet most of our profits on this and are committed to extending CUDA from GeForce to every computer. The reason we are so determined is because we believe in its potential. Despite initial hardships, the company held on to this belief for 13 generations and 20 years, and today CUDA is everywhere.

It was pixel shaders that fueled the GeForce revolution. And about eight years ago, we launched RTX — a complete architectural revolution for the modern age of computer graphics. GeForce brought CUDA to the world, and because of this, many scholars such as Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton, and Andrew Ng discovered that GPUs can be a powerful tool to accelerate deep learning, thus igniting the explosion of artificial intelligence ten years ago.

Ten years ago, we decided to combine programmable shading with two new ideas: hardware ray tracing (Ray Tracing), which was technically challenging; and a forward-looking idea at the time — about a decade ago, we anticipated that AI would completely transform computer graphics. Just as GeForce brought AI to the world, AI will now, in turn, reshape the way entire computer graphics are implemented.

Today, I want to show you the future. This is our next generation graphics technology, and we call it Neural Rendering (Neural Rendering) — a deep fusion of 3D graphics and artificial intelligence. This is DLSS 5, take a look.

Neural rendering: the fusion of structured data and generative AI

Isn't that amazing? As a result, computer graphics came to life.

What have we done? We combine controllable 3D graphics (the real foundation of the virtual world) with its structured data, and incorporate generative AI and probabilistic computation. One is completely deterministic, the other is probabilistic yet highly realistic — we combine these two concepts to achieve accurate and controllable structured data while generating it in real time. At the end of the day, the content is both beautiful and stunning, and completely controllable.

The concept of integrating structured information with generative AI will continue to be replicated in industry after industry. Structured data is the cornerstone of trusted AI.

Accelerated platform for structured and unstructured data

Now I'm going to take you to a technical architecture diagram.

Structured data - Familiar SQL, Spark, Pandas, Velox, and important platforms such as Snowflake, Databricks, Amazon EMR, Azure Fabric, and Google BigQuery all process data frames (Data Frames). These data frames are like giant spreadsheets that carry all the information in the business world and are the basic facts (ground truth) of enterprise computing.

In the age of AI, we need AI to use structured data and accelerate it to the extreme. In the past, accelerated structured data processing was meant to make businesses run more efficiently. In the future, AI will use these data structures at a speed far exceeding that of humans, and AI agents will also make extensive calls to structured databases.

In terms of unstructured data, vector databases, PDF, video, audio, etc. make up the vast majority of data forms in the world — about 90% of the data generated each year is unstructured data. In the past, this data was almost completely unusable: we read it, stored it in the file system, and that was all. We can't search and search because unstructured data lacks simple indexing methods, and we must understand its meaning and context. Now, AI can do that—with multi-modal perception and understanding technology, AI can read PDF documents, understand their meaning, and embed them within a larger structure that can be queried.

Nvidia has created two basic libraries for this:

• cuDF: for accelerated processing of data frames and structured data

• CUVs: for vector storage, semantic data, and processing of unstructured AI data

These two platforms will be one of the most important basic platforms of the future.

Today, we are announcing partnerships with a number of companies. IBM, the inventor of the SQL language, will use CUDF to accelerate its WatsonX Data platform. Dell co-created the Dell AI data platform with us, integrated CUDF and CUVs, and achieved significant performance improvements in NTT Data's actual projects. On the Google Cloud side, we're now not only speeding up Vertex AI, but also accelerating BigQuery, and partnering with Snapchat to reduce its computational costs by nearly 80%.

The benefits of accelerated computing are trifecta: speed, scale, and cost. This is in line with the logic of Moore's Law — achieving a leap in performance through accelerated computation while continuously optimizing the algorithm, so that everyone can enjoy the continuously declining computing costs.

Nvidia has built an accelerated computing platform with many libraries: RTX, CUDF, CUvs, etc. These libraries are integrated into global cloud services and OEM systems to reach global users.

In-depth cooperation with cloud service providers

Partnering with major cloud service providers

Google Cloud: We accelerate Vertex AI and BigQuery, deeply integrate with JAX/XLA, and excel on PyTorch—Nvidia is the only accelerator in the world that excels on both PyTorch and JAX/XLA. We are bringing customers such as Base10, CrowdStrike, Puma, and Salesforce to the Google Cloud ecosystem.

AWS: We accelerate EMR, SageMaker, and Bedrock, and are deeply integrated with AWS. I am particularly excited this year that we will be introducing OpenAI to AWS, which will dramatically increase the consumption of AWS cloud computing and help OpenAI expand regional deployment and computing scale.

Microsoft Azure: The Nvidia 100 PFLOPS supercomputer was the first supercomputer we built and the first supercomputer deployed on Azure, which provided an important foundation for our partnership with OpenAI. We accelerate Azure cloud services and AI Foundry, work together to expand Azure regions, and collaborate deeply on Bing search. It's worth mentioning that our**Confidential Computing (Confidential Computing) ** capability — ensuring that even operators can't view user data and models — Nvidia GPUs are the world's first GPUs to support confidential computing, and can support the confidential deployment of OpenAI and Anthropic models in cloud environments in various regions around the world. Using Synopsys as an example, we accelerated all of their EDA and CAD workflows and deployed on Microsoft Azure.

Oracle: We are Oracle's first AI customer, and I'm proud to be able to explain the AI cloud concept to Oracle for the first time. Since then, they've grown rapidly, and we've brought in many partners such as Cohere, Fireworks, and OpenAI.

CoreWeave: The world's first AI-native cloud, created specifically for GPU hosting and AI cloud services, has an excellent customer base and is growing strongly.

Palantir + Dell: The three parties have jointly created a new AI platform. Based on Palantir's Ontology Platform (Ontology Platform) and AI platform, AI can be deployed completely locally in any country and in any air gap isolation environment — everything from data processing (vectorization or structuring) to a complete accelerated computing stack for AI.

Nvidia has established this special partnership with global cloud service providers — we bring our customers to the cloud, which is a mutually beneficial and win-win ecosystem.

Vertical integration, horizontal openness: Nvidia's core strategy

Nvidia is the world's first vertically integrated and horizontally open company.

The need for this model is very simple: accelerated computing is not a chip issue, nor a system issue; its full description should be application acceleration. The CPU can make the computer run faster overall, but this path has reached a bottleneck. In the future, only application-specific acceleration can continue to bring about leaps in performance and cost reductions.

This is why Nvidia must deeply cultivate one library after another, one field after another, and one vertical industry after another. We're a vertically integrated computing company, and there's no other way. We must understand the application, understand the field, deeply understand the algorithm, and be able to deploy it in any scenario — data center, cloud, local, edge, and even robotic systems.

At the same time, Nvidia remains horizontally open and is willing to integrate technology into any partner's platform, so that the world can enjoy the dividends of accelerated computing.

This is fully reflected in the structure of participants at this GTC. Among the participants this time, the financial services industry had the highest proportion - developers, not traders, wanted to come. Our ecosystem covers both the upstream and downstream supply chains. Whether the company was founded 50, 70, or 150 years ago, last year had its best year in history. We're at the beginning of something very, very important.

CUDA-X: Accelerated Computing Engine for Various Industries

In every vertical field, Nvidia has developed an in-depth layout:

• Autonomous driving: broad coverage and far-reaching impact

• Financial services: quantitative investment is shifting from artificial feature engineering to supercomputer-driven deep learning, ushering in its “Transformer Moment”

• Healthcare: We are ushering in our own “ChatGPT moment”, covering areas such as AI-assisted drug discovery, AI agent support diagnosis, and medical customer service

• Industry: The world's largest construction wave is underway, and AI factories, chip factories, and data center factories have been launched one after another

• Entertainment and gaming: Real-time AI platform supports translation, live streaming, game interaction, and smart shopping agents

• Robotics: Deeply cultivated for more than ten years, the three major computer architectures (training computers, simulation computers, airborne computers) are complete. A total of 110 robots were unveiled at this exhibition

• Telecom: In an industry worth about $2 trillion, base stations will evolve from a single communication function to an AI infrastructure platform. The related platform is called Aerial, and there is deep cooperation with companies such as Nokia and T-Mobile

At the heart of all of these areas is our CUDA-X library — it's the foundation of Nvidia as an algorithm company. These libraries are the company's core assets, enabling computing platforms to exert real value in various industries.

One of the most important libraries is cuDNN (CUDA Deep Neural Network Library), which completely revolutionized artificial intelligence and sparked the explosion of modern AI.

(Play CUDA-X demo video)

Everything you've just seen is simulation — including physics-based solvers, AI agent physical models, and physical AI robot models. Everything is simulated, without any manual animations or joint bindings. This is Nvidia's core competency: unlocking these opportunities through a deep understanding of algorithms and organic integration of computing platforms.

AI Native Enterprise and the New Era of Computing

You've just seen the industry giants that define today's society, such as Walmart, L'Oréal, J.P. Morgan, Roche, and Toyota, as well as a number of companies you've never heard of — we call them AI-native companies. The list is huge, and includes OpenAI, Anthropic, and a host of start-ups serving different verticals.

The industry has experienced an amazing take-off over the past two years. Venture capital inflows to startups reached $150 billion, the largest amount in human history. More importantly, for the first time, the size of a single investment jumped from millions of dollars to hundreds of millions or even billions of dollars. There's only one reason: for the first time in history, every company of this kind requires a lot of computing resources and lots of tokens. The industry is creating, generating, or adding value to tokens from organizations such as Anthropic and OpenAI.

Just as the PC revolution, the Internet revolution, and the mobile cloud revolution have each spawned a number of epoch-making enterprises, this generation of computing platform transformation will also give birth to a number of highly influential companies and become an important force in the future world.

Three historic breakthroughs that drove it all

What really happened in the past two years? Three important things.

First: ChatGPT, ushering in the generative AI era (end 2022 to 2023)

It can not only sense and understand, but also generate unique content. I showed the fusion of generative AI and computer graphics. Generative AI has fundamentally changed the way computing—from retrieval to generative computation, which profoundly affects computer architectures, deployment methods, and overall significance.

Second: Inference AI (Inference AI), represented by o1

The ability to reason enables AI to self-reflect, plan, and break down problems — breaking down problems it can't directly understand into actionable steps. o1 makes generative AI trustworthy and can make inferences based on real information. As a result, the amount of tokens in the input context and the amount of tokens output for thought increased dramatically, and the amount of computation increased significantly.

Third: Claude Code, the first smart body model

It can read files, write code, compile, test, evaluate, and iterate. Claude Code has revolutionized software engineering—100% of Nvidia engineers use one or more of Claude Code, Codex, and Cursor, and no software engineer isn't helped by AI.

This is a new inflection point — instead of asking AI “what, where, and how to do it,” you let it “create, execute, build” and let it actively use tools, read files, break down problems, and act on it. From perception, to generation, to reasoning, AI can actually get the job done today.

Over the past two years, the amount of computation required for inference has increased about 10,000 times, and usage has increased about 100 times. I've always believed that computing demand has grown 1 million times over the past two years—this is everyone's feeling, OpenAI's feeling, and Anthropic's feeling. If more computing power can be obtained, more tokens can be generated, revenue will increase, and AI will become more intelligent. The inflection point in reasoning has arrived.

The trillion-dollar AI infrastructure era

At this time last year, I was here to say that we have high confidence in Blackwell and Rubin's demand and purchase orders until 2026, which is approximately $500 billion. Today, one year after GTC, I'm standing here to tell you: Looking ahead to 2027, I'm seeing a figure of at least $1 trillion. And I'm sure the actual computational requirements will go far beyond that.

2025: Nvidia's Year of Reasoning

2025 is Nvidia's Year of Inference (Year of Inference). We want to ensure that beyond training and post-training, we can maintain excellence at every stage of the AI lifecycle, so that the invested infrastructure can continue to operate efficiently, and the longer the effective service life, the lower the unit cost.

Meanwhile, Anthropic and Meta have officially joined the NVIDIA platform, which together represents one-third of the world's AI computing power requirements. The open source model is close to the cutting edge and is everywhere.

Nvidia is currently the only platform in the world that can run all AI models in all fields — language, biology, computer graphics, computer vision, speech, protein and chemistry, robotics, etc. — regardless of language, whether edge or cloud. The Nvidia architecture is universal for all of these scenarios, making us the lowest-cost and most reliable platform.

Currently, 60% of Nvidia's business comes from the world's top five hyperscale cloud service providers, and the remaining 40% are in various fields such as regional clouds, sovereign clouds, enterprises, industry, robotics, and edge computing. The breadth of AI coverage itself is where its resilience lies — this is undoubtedly a new transformation of computing platforms.

Grace Blackwell and NVLink 72: A bold architectural revolution

While the Hopper architecture was still in its heyday, we decided to completely restructure the system, expand NVLink from 8 channels to NVLink 72, and completely decompose the computing system. Grace Blackwell NVLink 72 was a huge technology bet, and it wasn't easy for any of our partners, and I would like to thank everyone sincerely.

At the same time, we launched NVFP4 — not just an ordinary FP4, but a new kind of tensor core and compute unit. We have proven that the NVFP4 can perform inference without loss of accuracy, while providing significant performance and energy efficiency improvements, and is also suitable for training. Furthermore, a series of new algorithms, such as Dynamo and TensorRT-LLM, have been introduced one after another, and we have even dedicated billions of dollars to build a supercomputer called DGX Cloud to optimize the kernel.

The results proved that our reasoning performance was impressive. Data from Semi Analysis—the most comprehensive AI inference performance measurement—shows that Nvidia is far ahead in terms of both the number of tokens per watt and the cost per token. Originally Moore's Law might have given the H200 a 1.5 times performance boost, but we did it 35 times. Semi Analysis' Dylan Patel even said, “Hwang In-hoon is conservative; it's actually 50 times more.” He was right.

I'm quoting him here: “Jensen sandbagged (Hwang In-hoon's conservative score).”

Nvidia's cost per token is the lowest in the world, and currently no one can match it. The reason is Extreme Co-design (Extreme Co-design).

Taking Fireworks as an example, before Nvidia updated its full suite of software and algorithms, its average token speed was about 700 per second; after the update, it was close to 5,000 per second, an increase of about 7 times. This is the ultimate power of collaborative design.

AI factory: from data center to token factory

Data centers used to be places to store files; now they are factories that produce tokens. Every cloud service provider and every AI company will use “token factory efficiency” as the core business indicator in the future.

This is my core argument:

• Vertical axis: Throughput — number of tokens generated per second at fixed power

• Horizontal axis: Interaction speed (Token Speed) - the response speed for each inference. The faster the speed, the larger the model that can be used, the longer the context, and the more intelligent the AI

Tokens are new commodities, and once mature, they will be tiered for pricing:

• Free tier (high throughput, low speed)

• Mid-tier (~$3 per million tokens)

• Premium tier (~$6 per million tokens)

• High speed tier (~$45 per million tokens)

• Ultra High Speed Tier (~$150 per million tokens)

Compared to Hopper, Grace Blackwell increased throughput by 35 times at the highest value tier and introduced a new tier. Using simplified model estimates, with 25% of the power distributed to each of the four tiers, Grace Blackwell could generate 5 times more revenue than Hopper.

Vera Rubin: Next Generation AI Computing System

(Play Vera Rubin System introduction video)

Vera Rubin is a complete, end-to-end optimized system designed for agentic workloads:

• Large language model computing core: NVLink 72 GPU cluster, prefill (prefill) and KV Cache

• New Vera CPU: Designed for extremely high single-threaded performance, uses LPDDR5 memory, and has excellent energy efficiency. It is the only data center CPU in the world that uses LPDDR5, suitable for AI agent tool calls

• Storage system: BlueField 4 + CX 9, a new storage platform for the AI era, 100% participation in the global storage industry

• CPO Spectrum X Switch: The world's first co-packaged optical Ethernet switch, fully mass-produced

• Kyber rack: A new rack system that supports 144 GPUs to form a single NVLink domain, front-end computing and back-end NVLink switching to form a giant computer

• Rubin Ultra: Next-generation supercomputing node, vertical plug-in design, with Kyber racks to support larger NVLink interconnections

Vera Rubin is 100% liquid cooled, reducing installation time from two days to two hours, using 45°C hot water cooling, which greatly reduces data center cooling pressure. This time, Satya (Nadella) has published an article confirming that the first Vera Rubin rack is now running on Microsoft Azure, and I am very excited about this.

Groq integration: the ultimate extension of inference performance

We bought the Groq team and licensed their technology. Groq is a deterministic dataflow processor (deterministic dataflow processor). It uses static compilation and compiler scheduling, has a large amount of SRAM, is optimized for a single inference workload, and has extremely low latency and extremely high token generation speed.

However, Groq's limited memory capacity (500MB on-chip SRAM) makes it difficult to independently carry large model parameters and KV Cache, limiting its large-scale application.

The solution is Dynamo — a suite of inference and scheduling software. We use Dynamo to disaggregate the inference pipeline (Disaggregate):

• **Prefill (Prefill) and decode the attention mechanism (Decode) ** completed on Vera Rubin (requires a lot of computing power and KV Cache storage)

• **Feedforward Network Decode (Feed-Forward Network Decode) ** means the token generation part, done on Groq (requires extremely high bandwidth and low latency)

The two are tightly coupled via Ethernet, and the delay is reduced by about half thanks to a special mode. Under the unified scheduling of Dynamo's “AI factory operating system,” overall performance was increased 35 times, and a new level of inference performance that NVLink 72 could not reach before was opened.

Groq and Vera Rubin's combination suggestions:

• If your workload is mostly high throughput, use 100% Vera Rubin

• If a large workload generates high-value tokens such as code generation, Groq can be introduced. The recommended ratio is about 25% Groq + 75% Vera Rubin

The Groq LP30 is manufactured by Samsung and is currently in mass production. Shipments are expected to begin in Q3. Thank you Samsung for your full cooperation.

A historic leap forward in inference performance

Quantifying previous technological advancements: in 2 years, the 1-gigawatt AI factory's token generation rate will increase 350 times from 22 million tokens/s to 700 million tokens/s. This is the ultimate power of collaborative design.

Technology roadmap

• Blackwell: Currently in production, Oberon standard rack system, copper extension to NVLink 72, optional optical extension to NVLink 576

• Vera Rubin (current): Kyber rack, NVLink 144 (copper); Oberon rack, NVLink 72 + optical, extended to NVLink 576; Spectrum 6, the world's first CPO switch

• Vera Rubin Ultra (coming soon): Next-generation Rubin Ultra GPU, LP35 chip (first integrated with NVFP4), further boosts performance by several times

• Feynman (next generation): New GPU, LP40 chip (co-built by Nvidia and Groq teams, integrated NVFP4); new CPU—Rosalyn (Rosalyn); BlueField 5; CX 10; Kyber rack supporting both copper and CPO expansion methods

The road map is clear: the three routes of copper cable expansion, optical expansion (scale-up), and optical expansion (scale-out) are progressing in parallel, and we need all partners to continue to expand production in terms of copper cable, fiber, and CPO.

NVIDIA DSX: Digital Twin Platform for AI Factories

The AI factory is getting more complex, but the various technology vendors that make it up have never collaborated with each other in the design phase until they “met” in the data center — which is clearly not enough.

To that end, we've created Omniverse, and the NVIDIA DSX platform based on it — a platform for all partners to co-design and operate gigawatt AI factories in a virtual world. DSX offers:

• Rack-level mechanical, thermal, electrical, and network simulation systems

• Connect to the power grid to achieve collaborative energy saving scheduling

• Max-Q-based dynamic power and cooling optimization in data centers

It is conservatively estimated that this system can increase energy efficiency by about 2 times, which is a very significant benefit on the scale we're talking about. Starting with a digital Earth, Omniverse will host digital twins of all sizes, and we're working with partners around the world to build the largest computer in human history.

Furthermore, Nvidia is entering space. The Thor chip has been radiation-certified and is operating in a satellite. We are developing Vera Rubin Space-1 with our partners to build a space data center. In space, we can only rely on radiation to dissipate heat. Thermal management is a core challenge, and we are gathering top engineers to tackle it.

OpenClaw: Operating System for the Smart Era

Peter Steinberger developed a piece of software called OpenClaw. It's the most popular open source project in human history, surpassing the achievements of three decades of Linux in just a few weeks.

OpenClaw is essentially an Agentic System (Agentic System) that can:

• Manage resources, access tools, file systems, and large language models

• Execute scheduled and scheduled tasks

• Break down the problem step by step and call the sub-agents

• Supports any mode of input and output (voice, video, text, email, etc.)

Described in the syntax of an operating system, it really is an operating system — the operating system of an intelligent computer. Windows makes personal computers possible, and OpenClaw makes personal intelligence possible.

Every business needs to develop its own OpenCLAW strategy, just as we all need a Linux strategy, an HTML strategy, and a Kubernetes strategy.

Complete reshaping of enterprise IT

Enterprise IT before OpenClaw: data and files enter systems, flow through tools and workflows, and eventually become tools for human use. Software companies create tools, systems integrators (GSIs), and consulting firms help businesses use these tools.

Enterprise IT after OpenClaw: Every SaaS company will be transformed into an AaaS (Agentic as a Service) company — not just providing tools, but AI agents that specialize in specific fields.

But here's a key challenge: intelligence within the enterprise can access sensitive data, execute code, and communicate with the outside world. This must be strictly controlled in an enterprise environment.

To this end, we've partnered with Peter to incorporate security into the enterprise-grade version to launch:

• NeMo Claw (reference design): An enterprise-grade reference framework based on OpenClaw that integrates NVIDIA's full suite of intelligent AI toolkits

• Open Shield (security layer): Integrated into OpenClaw to provide a policy engine, network fence, and privacy routing to ensure enterprise data security

• NeMo Cloud: Downloadable and connected to the policy engine of any SaaS enterprise

This is the renaissance of enterprise IT. What was originally a $2 trillion industry will soon grow to a scale of trillions of dollars, shifting from providing tools to providing specialized AI intelligent services.

I can fully anticipate that in the future, every engineer in the company will have an annual token budget. Their annual salary is probably hundreds of thousands of dollars, and I will give them an additional token quota equal to half of their salary to increase their output tenfold.” “How many token quota comes with onboarding” has become a new recruitment topic in Silicon Valley.

Every enterprise in the future will be both a token user (for engineers) and a token producer (to provide services to its customers). The significance of OpenClaw cannot be underestimated; it is as important as HTML and Linux.

NVIDIA Open Model Initiative

In terms of custom claws (Custom Claw), we provide cutting-edge models developed in-house by NVIDIA:

Model Field Nemotron Large Language Model Cosmos World Foundation Model (World Foundation Model) GROOT Universal Humanoid Robot Model Alpamayo Autonomous Driving BioneMo Digital Biology Phys-AIAI Physics

We are at the cutting edge of technology in every field and are committed to continuous iteration—Nemotron 3 is followed by Nemotron 4, Cosmos 1 is followed by Cosmos 2, and Groq is also iterating to the second generation.

Nemotron 3 is ranked among the three best models in the world in OpenCLAW and is at the cutting edge. Nemotron 3 Ultra will be the strongest basic model ever, supporting countries to build sovereign AI.

Today, we announced the establishment of the Nemotron Alliance to invest billions of dollars to advance the development of AI basic models. Alliance members include BlackForest Labs, Cursor, LangChain, Mistral, Perplexity, Reflection, Sarvam (India), Thinking Machines (Mira Murati's lab), etc. Enterprise software companies have joined in to integrate NeMo Claw reference designs and the NVIDIA Smart AI toolkit into their products.

Physical AI and robotics

Digital intelligence acts in a digital world — writing code and analyzing data; physical AI is an embodied intelligence, or robot.

This time, a total of 110 robots were unveiled at GTC, including almost all robot research and development companies in the world. Nvidia provides three computers (training computer, simulation computer, airborne computer) and a complete software stack and AI model.

In terms of autonomous driving, the “ChatGPT moment” for autonomous driving has arrived. Today, we're announcing the addition of four new partners to Nvidia's RoboTaxi Ready platform: BYD, Hyundai, Nissan, and Geely, with a combined annual production capacity of 18 million vehicles. In addition to the previous Mercedes-Benz, Toyota, and GM, the lineup was further expanded. We also announced a major partnership with Uber to deploy and connect RoboTaxi Ready vehicles in multiple cities.

In terms of industrial robots, ABB, Universal Robotics, KUKA and many other robotics companies cooperate with us to combine physical AI models with simulation systems to promote the implementation of robots in global manufacturing production lines.

In terms of telecommunications, Caterpillar (Caterpillar) and T-Mobile are also among them. In the future, the wireless base station will no longer be just a communication node, but an NVIDIA Aerial AI RAN—an intelligent edge computing platform that can sense traffic in real time, adjust beamforming, and achieve energy efficiency.

Special session: Olaf robot unveiled

(Play Disney Olaf robot demo video)

Hwang In-hoon: A snowman is here! Newton works fine! Omniverse works fine too! Olaf, how are you?

Olaf: I'm really happy to see you.

Hwang In-hoon: Yes, because I gave you a computer—Jetson!

Olaf: What's that?

Wong In-hoon: Right in your stomach.

Olaf: That's amazing.

Wong In-hoon: You learned to walk in the Omniverse.

Olaf: I love walking. It's much better than riding a reindeer and looking at the beautiful sky.

Hwang In-hoon: That's because of physical simulation—the Newton solver running on NVIDIA Warp, which we developed in collaboration with Disney and DeepMind, so you can adapt to the real physical world.

Olaf: I was just trying to say that.

Hwang In-hoon: This is where you are smart. I'm a snowman, not a snowball.

Hwang In-hoon: Can you imagine that? The Disneyland of the future — all these robot characters roam freely in the park. But honestly, I thought you'd be taller. I've never seen such a short snowman.

Olaf: (disagree)

Huang Renxun: Can you help me finish today's speech?

Olaf: That's great!

Keynote Summaries

Huang Renxun: Today, we discussed the following core topics together:

1. The inflection point of inference has arrived: Inference has become the core workload of AI. Tokens are a new commodity, and inference performance directly determines revenue

2. AI factory era: Data centers have evolved from file storage facilities to token production plants. In the future, every company will use “AI factory efficiency” to measure its competitiveness

3. OpenClaw smart revolution: OpenClaw has begun the era of intelligent computing. Enterprise IT is moving from the age of tools to the age of smart devices, and every enterprise needs to develop an OpenClaw strategy

4. Physical AI and robotics: embodied intelligence is being implemented on a large scale, and autonomous driving, industrial robots, and humanoid robots together form the next major opportunity for physical AI

Thanks everyone and happy GTC!