On April 22, 2026, at Google Cloud Next, Google announced its eighth generation of Tensor Processing Units, and for the first time split a TPU generation into two different chips: TPU 8t for large-scale training and TPU 8i for inference and reinforcement learning. Google says TPU 8t delivers up to 2.7 times the training performance per dollar of Ironwood, its seventh-generation TPU, and that TPU 8i delivers up to 80% better inference performance per dollar. The split is a hardware bet that agentic AI, with long contexts, Mixture-of-Experts (MoE) models and many sequential reasoning steps, needs a different machine than pre-training does.

Key Facts#

  • Announced: April 22, 2026, at Google Cloud Next, as part of Google's AI Hypercomputer.
  • Two chips: TPU 8t targets pre-training and embedding-heavy workloads, while TPU 8i targets sampling, serving, reasoning and reinforcement learning.
  • TPU 8t scale: 9,600 chips per superpod, which Google says provides 121 exaflops and two petabytes of shared memory. Each chip has 216 GB of HBM and a listed peak of 12.6 petaflops at FP4 precision.
  • TPU 8i memory: 384 MB of on-chip SRAM (three times the previous generation) and 288 GB of HBM, so larger KV caches can stay on the chip.
  • Efficiency claims: up to 2.7x better training performance per dollar (TPU 8t) and up to 80% better inference performance per dollar (TPU 8i) than Ironwood, plus up to 2x better performance per watt for both, according to Google.
  • Cluster scale: the new Virgo Network fabric links 134,000 TPU 8t chips in one data center, and training clusters can span more than one million chips across sites.
  • Software: native PyTorch support for TPUs entered preview, alongside JAX, vLLM and XLA.
  • Availability: Google said both systems "will be available to Cloud customers soon" and opened an interest form.

What Happened#

Google framed the launch around a claim that training and serving have diverged. Pre-training rewards raw throughput across enormous clusters. Serving reasoning models rewards low latency, memory close to the compute, and fast all-to-all communication, because an MoE model may route any token to an expert on any chip.

TPU 8t is the throughput machine. It keeps the 3D torus topology of earlier TPUs at a larger scale and adds native 4-bit floating point (FP4), which Google says doubles the throughput of the matrix units while preserving accuracy for large models. A SparseCore accelerator handles embedding lookups, and new TPUDirect RDMA and TPUDirect Storage paths move data between TPU memory, network cards and storage without passing through the host CPU. With Managed Lustre, Google claims 10 times faster storage access than training on Ironwood.

TPU 8i is the latency machine. Its larger SRAM lets a long-context KV cache live on the chip, which reduces idle time during decoding. A new Collectives Acceleration Engine (CAE), which replaces the four SparseCores of an Ironwood chip, aggregates results across cores and cuts on-chip collective latency by up to five times. TPU 8i also drops the torus for a topology Google calls Boardfly: four-chip boards are fully connected into groups of eight, and 36 groups are linked through optical circuit switches into a pod of up to 1,152 chips, 1,024 of them active. Google's own arithmetic shows why. In a 1,024-chip 3D torus, the farthest chip is 16 hops away, and Boardfly cuts that to seven, a 56% reduction in network diameter. Google says this yields up to 50% lower latency for communication-heavy workloads.

Both systems use Google's Arm-based Axion CPUs as hosts, and Google announced them alongside A5X bare-metal instances based on NVIDIA's Vera Rubin NVL72.

Background#

Ironwood (TPU v7x), Google's seventh-generation TPU, is the baseline for all of the new efficiency claims. In October 2025, Anthropic said it would expand its use of Google Cloud to as many as one million TPUs, bringing well over a gigawatt of capacity online in 2026, in a deal worth tens of billions of dollars. Google Cloud CEO Thomas Kurian cited Ironwood in that announcement. On April 6, 2026, Anthropic went further, signing an agreement with Google and Broadcom for multiple gigawatts of next-generation TPU capacity expected to start coming online in 2027. Anthropic did not say which TPU generation that capacity will use.

Google says the new chips were designed with Google DeepMind to train and serve world models such as Genie 3 as well as reasoning agents that run many dependent steps per request.

Why It Matters for Developers#

Few .NET developers will program a TPU directly. The software stack Google lists, including JAX, PyTorch, vLLM and XLA, is Python-centric, and most .NET teams will feel TPUs through the prices, latency and context limits of hosted models they call over HTTP.

  • Inference economics shape your design. Agentic features make many model calls per user request, so cost per token and time to first token decide whether a design is viable. If Google's inference claims show up in model pricing, long-context and multi-step agents become cheaper to run. Our guide to AI agent architecture patterns covers how to keep call chains short and parallel where possible.
  • Keep your code provider-neutral. Hardware generations shift the price-performance ranking of models and clouds every year. Coding against IChatClient, as described in our Microsoft.Extensions.AI guide, lets you move workloads between models and providers without rewriting business logic.
  • Measure before and after migrations. Treat vendor efficiency numbers as claims until your own telemetry confirms them. Track tokens, latency and cost per feature with OpenTelemetry, as shown in our LLM observability and cost guide.
  • Watch the PyTorch preview if you fine-tune. Teams that fine-tune models in Python and serve them to .NET applications often default to GPUs because their code already runs there. Native PyTorch support lowers the cost of trying TPUs.

The Boardfly and SRAM choices also carry a lesson for architects: at scale, latency is dominated by data movement, not arithmetic. One level up, caching prompts and keeping context compact often matter more than picking a faster model.

What's Next#

Google did not publish prices or a general availability date for TPU 8t and TPU 8i at launch. It said only that they would reach Cloud customers "soon" and invited interested customers to register. Native PyTorch support for TPUs is in preview with select customers, and Google said it expects to be among the first clouds to offer NVIDIA Vera Rubin instances when that platform becomes available later in 2026.

Open questions include regional availability, whether independent benchmarks confirm the performance-per-dollar claims, and how much capacity goes to large customers such as Anthropic. For a competing design, see our coverage of Microsoft's Maia 200 inference accelerator.

Sources#