Why AI Agent Growth Could Shift Computing Demand From GPUs to CPUs

As autonomous agents multiply and push massive volumes of interactive traffic across global networks, the artificial intelligence industry may soon pivot away from its heavy reliance on centralized graphics processing units.

Adam Karon, chief operating officer and general manager of the Cloud Technology Group at Akamai, reported that computing percentages will shift from GPUs toward CPUs over time as automated systems take on everyday tasks at the network edge.

The Tech TL;DR:

• Akamai COO Adam Karon forecasts that growing numbers of autonomous AI agents will shift regular workloads away from costly centralized GPUs onto CPUs.
• Running agent tasks and decision APIs on conventional processors at network edge locations cuts down latency and avoids the high costs of data center accelerators.
• While massive GPU clusters remain critical for training frontier models, distributed CPU infrastructure offers a more economical approach for routine enterprise API execution.

While massive GPU clusters remain essential for training frontier models in centralized facilities, Karon explained that the operational dynamics change dramatically once those models leave the laboratory.

Autonomous agents now function as active internet users by retrieving data, interacting with enterprise systems, executing code, and making API calls. Routing every single interaction back to a distant data center introduces severe latency penalties and mirrors the inefficiencies of legacy mainframe computing models.

Distributed infrastructure helps mitigate these performance bottlenecks by bringing computing power closer to the end user. Akamai currently operates infrastructure across more than 4,000 edge locations globally, allowing organizations to run lightweight agent workflows directly on conventional processors. Karon noted that intelligent architectures require a hybrid division of labor where GPUs handle heavy parallel inference while CPUs manage routing, policy enforcement, and operational state management.

The underlying cloud economics heavily favor this balanced approach for routine enterprise tasks. An AWS cloud instance equipped with a single Nvidia H100 GPU costs roughly $6.88 per hour on-demand, whereas a 64-vCPU AWS Graviton3 CPU instance costs approximately $2.32 per hour. Although these configurations serve entirely different workloads, the stark pricing delta highlights why developers should avoid allocating expensive accelerated hardware to standard business logic that conventional processors can execute efficiently.

Autonomous agents also generate massive volumes of continuous network chatter as they communicate repeatedly with external databases and other automated systems. Karon likened this behavior to the character Donkey in the movie Shrek, pointing out that these agents do not know when to stop talking. Transmitting every minor conversational exchange across wide-area networks strains telecom infrastructure unless developers push compute resources directly to the network edge where applications actually reside.

To reduce unnecessary overhead, modern engineering teams increasingly rely on specialized decision APIs rather than forcing large language models to reason through minor operational choices.

Developers use streamlined models provided by platforms such as OpenAI, Microsoft, Perplexity, and Cloudflare to return choices, scores or probabilities rather than lengthy generated responses. Implementing these decision-making layers on localized CPU infrastructure allows enterprises to scale automated workflows without incurring the recurring cost penalties of centralized GPU clusters.

Always consult with certified IT and cybersecurity professionals before altering enterprise networks or handling sensitive data.