SemiAnalysis (@semianalysis.com.web.brid.gy) Bsky

xAI’s Colossus 2 – First Gigawatt Datacenter In The World, Unique RL Methodology, Capital Raise Much has been written about xAI’s Colossus 1. The Memphis build belongs in the history books: the largest AI training cluster, erected from scratch in 122 days. With roughly 200,000 H100/H200s and ~30,000 GB200 NVL72, it remains, today, the largest fully operational, single-coherent cluster (setting apart Google, master of multi-datacenter-training). However, Colossus 1’s ~300 MW […]

7 months ago 0 0 0 0

Another Giant Leap: The Rubin CPX Specialized Accelerator & Rack Nvidia announced the Rubin CPX, a solution that is specifically designed to be optimized for the prefill phase, with the single-die Rubin CPX heavily emphasizing compute FLOPS over memory bandwidth. This is a game changer for inference, and its significance is surpassed only by the March 2024 announcement of the GB200 NVL72 Oberon rack-scale form […]

7 months ago 0 0 0 0

Huawei Ascend Production Ramp: Die Banks, TSMC Continued Production, HBM is The Bottleneck Compute is the lifeblood of AI. He who controls the spice controls the universe the compute will control the production of tokens and reap the benefits of AI. Without compute you do not have a seat at the table. The United States technology community is all in on compute and AI as the next platform […]

7 months ago 0 0 0 0

Amazon’s AI Resurgence: AWS & Anthropic’s Multi-Gigawatt Trainium Expansion Two-and-a-half years ago, we flagged a looming “cloud crisis” at AWS. Today, the evidence has mounted. AWS is the crown jewel of the Amazon empire, generating ~60% of group profits, and dominating the lucrative Cloud Computing market. But it struggles to translate this strength into the new GPU/XPU Cloud era. Microsoft Azure now leads the […]

7 months ago 0 0 0 0

H100 vs GB200 NVL72 Training Benchmarks – Power, TCO, and Reliability Analysis, Software Improvement Over Time Frontier model training has pushed GPUs and AI systems to their absolute limits, making cost, efficiency, power, performance per TCO, and reliability central to the discussion on effective training. The Hopper vs Blackwell comparisons are not as simple as Nvidia would have you believe. In this report, we will start by present the results of […]

8 months ago 0 0 0 0

GPT-5 Set the Stage for Ad Monetization and the SuperApp To many power users (Pro and Plus), GPT5 was a disappointing release. But with closer inspection, the real release is focused on the vast majority of ChatGPT’s users, which is the 700m+ free userbase that is growing rapidly. Power users should be disappointed; this release wasn’t for them. The real consumer opportunity for OpenAI lies […]

8 months ago 0 0 0 0

Scaling the Memory Wall: The Rise and Roadmap of HBM The first portion of this report will explain HBM, the manufacturing process, dynamics between vendors, KVCache offload, disaggregated prefill decode, and wide / high-rank EP. The rest of the report will dive deeply into the future of HBM. We will cover the revolutionary change coming to HBM4 with custom base dies for HBM, what various […]

8 months ago 0 0 0 0

Robotics Levels of Autonomy Robots have powered manufacturing for decades, yet they stayed single-purpose and thrived only in perfect settings. Previous attempts at intelligent machines overpromised and underdelivered. But they were too early. Today, modern AI paradigms convert most robot roadblocks into data problems and push machines toward capabilities once thought impossible. As these models absorb real-world experience, robots […]

8 months ago 0 1 0 0

Intel 18A Details & Cost, Future of DRAM 4F2 vs 3D, Backside Power Adoption (or Not), China’s FlipFET, Digital Twins from Atoms to Fabs, and More Long time readers will recall that SemiAnalysis covers more than just datacenters and AMD. Today we’re back to semiconductors with a tech-focused roundup of the best from this year’s VLSI conference, the premiere design and integration. That includes the latest in chips manufacturing: fab digital twins, the future of advanced logic transistors and interconnects, DRAM […]

8 months ago 0 0 0 0

Meta Superintelligence – Leadership Compute, Talent, and Data Meta’s shocking purchase of 49% of Scale AI at a ~$30B valuation shows that money is of no concern for the $100B annual cashflow ad machine. Despite seemingly unlimited resources, Meta has been falling behind foundation labs in model performance.

9 months ago 0 0 0 0

Introduction to AI Networking Model Check out this short webinar where Dan and Patrick — the minds behind it — introduce our new AI Networking Model.

9 months ago 0 0 0 0

DeepSeek Debrief: >128 Days Later SemiAnalysis is hiring an analyst in New York City for Core Research, our world class research product for the finance industry. Please apply here It’s been a bit over 150 days since the launch of the Chinese LLM DeepSeek R1 shook stock markets and the Western AI world. R1 was the first model to be publicly […]

9 months ago 0 0 0 0

How Oracle Is Winning the AI Compute Market Oracle’s Cloud Infrastructure business is firing on all cylinders and is greatly outpacing expectations. All eyes are on the high-profile Stargate JV and the massive Abilene, Texas datacenter, which our September 2024 Multi-Datacenter Training report called out as a GW-scale training hub for OpenAI. But Oracle has many additional growth engines beyond this massive campus. […]

9 months ago 0 0 0 0

AI Training Load Fluctuations at Gigawatt-scale – Risk of Power Grid Blackout? The largest AI labs are racing to build multi-gigawatt-scale datacenters, and stressing our century-old power grid to an unprecedented extent. Not only is the scale massive, but AI training workloads have a very unique load profile, unexpectedly rising and falling from full load to nearly idle in fractions of a second. Our power grids were never designed […]

9 months ago 0 0 0 0

Ayar Labs | Co-packaged Optics Revolution | The Most Promising Hardware Startup With Wins At HPE And Nvidia? Ayar Labs is one of the most promising semiconductor startups in the semiconductor world. They were founded in 2015 with a team of many leading technologist from Intel, IBM, Micron, Penguin, MIT, Berkley, and Stanford. Ayar Labs saw one of the most fundamental problems in 2015 and started engineering a solution from the ground up. […]

9 months ago 0 0 0 0

NVIDIA Tensor Core Evolution: From Volta To Blackwell In our AI Scaling Laws article from late last year, we discussed how multiple stacks of AI scaling laws have continued to drive the AI industry forward, enabling greater than Moore’s Law growth in model capabilities as well as a commensurately rapid reduction in unit token costs. These scaling laws are driven by training and […]

9 months ago 0 0 0 0

AMD Advancing AI: MI350X and MI400 UALoE72, MI500 UAL256 For the past six months, AMD has been in a Wartime stance. They have been working hard and working smart towards their goal of being competitive with Nvidia. At its Advancing AI 2025 event, AMD launched the MI350X/MI355X GPUs which could be competitive to Nvidia’s HGX B200 solutions for inference of small to medium LLMs […]

10 months ago 0 0 0 0

The New AI Networks | Ultra Ethernet UEC | UALink vs Broadcom Scale Up Ethernet SUE Standard Ethernet intially lost significant market share to Nvidia’s InfiniBand in the early days of the GenAI boom. Since then, Ethernet has started clawing back market share, largely driven by cost, the various deficiencies of InfiniBand, as well as the ability to add more features and customization on top of Ethernet. Amazon and Google’s internal […]

10 months ago 0 0 0 0

Scaling Reinforcement Learning: Environments, Reward Hacking, Agents, Scaling Data The test time scaling paradigm is thriving. Reasoning models continue to rapidly improve, and are becoming more effective and affordable. Evaluations measuring real world software engineering tasks, like SWE-Bench, are seeing higher scores at cheaper costs. Below is a chart showing how models are both getting cheaper and better. Reinforcement learning (RL) is the reason […]

10 months ago 0 0 0 0

Posts by SemiAnalysis