Google Unveils TPU 8t and TPU 8i: Eighth-Generation AI Chips Built for Next-Gen Machine Learning Workloads

April 22, 2026 Rachel Kim – Technology Editor Technology

Google’s eighth-generation TPU family—TPU 8t and TPU 8i—lands in a market where model sizes are pushing past 10 trillion parameters and inference latency budgets for agentic workflows have tightened to sub-millisecond slices. The announcement, posted to the Google Cloud blog on April 20, 2026, positions these chips as the silicon backbone for next-gen autonomous agents that must reason, plan and act across multimodal streams without hitting a power or throughput wall. For platform engineers tasked with deploying large-scale LLM agents, the question isn’t whether to adopt custom ASICs but how to integrate them without reintroducing the vendor lock-in and thermal design headaches that plagued earlier TPU generations.

The Tech TL;DR:

  • TPU 8t delivers 4.5 PFLOPS bfloat16 peak with 1.2 TB/s HBM4 bandwidth, targeting training of 10T+ parameter models; TPU 8i offers 2.1 PFLOPS at 600 GB/s for low-latency agentic inference.
  • Both chips integrate a new SparseCore 3.0 engine that accelerates mixture-of-experts routing by 4.1x over TPU v5e, reducing activation memory footprint for MoE layers.
  • Google claims a 3.8x improvement in performance-per-watt over TPU v5e, measured via MLPerf Training v4.1 and Inference v3.0 benchmarks submitted under closed division.

The core architectural shift lies in the move from a monolithic MXU (Matrix Multiply Unit) to a disaggregated, tile-based compute fabric. Each TPU 8t die hosts 256 compute tiles interconnected via a 3D torus network with 25.6 Tb/s bisection bandwidth, a departure from the 2D mesh used in TPU v4. This enables near-linear scaling for tensor parallelism up to 1024 chips per pod, a critical factor for training frontier models where communication overhead has begun to dominate FLOPS utilization. The TPU 8i variant, optimized for inference, reduces tile count to 128 but adds a dedicated Agent Control Unit (ACU) that manages tool-use loops, function calling, and state persistence for agentic workloads—effectively turning the chip into a hardened runtime for ReAct-style agents.

According to the official TPU v8 architecture whitepaper, the chips are fabricated on a 3nm process node (likely TSMC N3E) and feature a new unified virtual address space (UVAS) that allows seamless migration of model shards between HBM4 and onboard SRAM without costly memcpy operations. This UVAS design, combined with the SparseCore 3.0, aims to alleviate the memory wall that has limited scaling efficiency in prior generations when dealing with sparsely activated expert networks.

“The real innovation isn’t the raw FLOPS—it’s how the ACU and UVAS let us treat the TPU as a stateful compute agent rather than a dumb matrix multiplier. We’ve cut agent loop latency from 12ms to 1.8ms on a 70B parameter MoE model running in FP8.”

— Lila Chen, Lead Architect for Agent Runtime, Google DeepMind (verified via internal tech talk, April 2026)

On the software front, Google has extended the XLA compiler with a new tf.agent.experimental.compile API that exposes the ACU’s instruction set for custom tool integration. Developers can now define agent-specific microcode via a domain-specific language (DSL) that compiles directly to the ACU’s RISC-V-like core. A sample workflow for a web-searching agent looks like this: 

 # Define agent toolchain via XLA agent DSL import tensorflow as tf from tensorflow.compiler.xla import agent @agent.tool(name="web_search", input_spec={"query": tf.string}) def search_web(query): # Custom ACU microcode for SERP parsing return agent.acu_call("web_search_v2", query) # Compile agent loop to TPU 8i strategy = tf.distribute.TPUStrategy(tpu_address="grpc://tpu-8i-pod:8470") with strategy.scope(): agent_model = tf.keras.Sequential([ tf.keras.layers.Embedding(32000, 4096), agent.ACUCell(num_tools=4, tool_specs=[search_web]) ]) agent_model.compile(optimizer="adam", loss="sparse_categorical_crossentropy")

This level of hardware-software co-design is where the TPU 8 series begins to diverge from general-purpose GPUs. While NVIDIA’s Blackwell architecture offers comparable raw throughput, its reliance on CUDA and TensorRT locks users into a software stack that introduces non-deterministic latency jitter—particularly problematic for agentic systems requiring hard real-time guarantees. The TPU’s deterministic execution model, enforced via static scheduling in XLA, provides a significant edge in scenarios where jitter must stay under 100 microseconds, such as robotic control loops or high-frequency trading agents.

From a deployment standpoint, the TPU 8 pods are now available via Google Cloud’s TPU VMs in the us-central1 and europe-west4 regions, with pricing starting at $4.20 per TPU 8t-hour and $2.10 per TPU 8i-hour (preemptible). However, the lack of multi-tenant isolation in the current release—pods are still dedicated per customer—means that enterprises with spiky workloads may face underutilization costs. This opens a niche for managed service providers who specialize in workload shaping and dynamic pod scheduling.

For organizations evaluating whether to refactor their agentic infrastructure around TPU 8, the immediate next step involves assessing network and storage bottlenecks. Even with 1.2 TB/s HBM4 bandwidth, feeding a 10T parameter model requires a storage subsystem capable of sustaining >100 GB/s read throughput during shard loading. This is where specialized cloud architecture consultants grow critical—they can design a parallel filesystem layer using Lustre or BeeGFS that avoids saturating the CIB (Chip-to-Interconnect Buffer) during model checkpoint restoration.

Security teams, meanwhile, must scrutinize the new attack surface introduced by the ACU’s programmable microcode. While Google has sandboxed the ACU via memory protection units and signed microcode blobs, the ability to load custom tool logic introduces a supply chain risk analogous to malicious CUDA kernels. Enterprises should engage code security auditors with expertise in firmware and microcode validation to audit any third-party agent toolchains before deployment.

Looking ahead, the TPU 8’s true test will come when agentic workloads begin to span multiple modalities—text, video, and sensor data—requiring heterogeneous compute. Google’s roadmap hints at a future TPU 9 variant that integrates a dedicated NPU for sensor fusion, but for now, the 8 series represents a pragmatic step toward overcoming the von Neumann bottleneck in agentic AI. The real metric to watch isn’t peak FLOPS but agent throughput per watt under sustained tool-use loops—a benchmark that Google has yet to publish but that will inevitably become the new yardstick for AI infrastructure.

*Disclaimer: The technical analyses and security protocols detailed in this article are for informational purposes only. Always consult with certified IT and cybersecurity professionals before altering enterprise networks or handling sensitive data.*

Worth a look