AI’s CPU Bottleneck: What Buyers Trade for Faster Deployment

An AI agent can finish a model call and still have work waiting: execute generated code inside an isolated sandbox, retrieve a document, call a tool, then assemble the results for another inference request. If those supporting services are queued, buying more GPU capacity does not clear the queue. The immediate constraint sits elsewhere.

That is the practical starting point for the AI CPU bottleneck—not a claim that every AI deployment is short of processors. Oracle’s October 27 announcement explicitly identifies agent orchestration, tool serving, sandbox execution and retrieval as workloads for NVIDIA Vera CPUs on OCI. These are the operations that turn a model response into a functioning service.

Arm’s November 4 results add a commercial question. As the architecture supplier moves toward delivering its own CPUs, buyers face different ways to obtain this capacity: commission custom Arm-based silicon, license a compute subsystem, or purchase a finished processor. Each places integration work—and control—somewhere different.

Who gets the shorter deployment schedule? Who controls the next upgrade?

The work between model calls

GPUs excel at the parallel arithmetic underpinning modern models. An agent’s surrounding workflow is less uniform. It involves scheduling processes, checking permissions, moving data and waiting for external services. CPUs provide general-purpose execution for much of that work, although a particular retrieval or tool pipeline may also use accelerators.

Consider a hypothetical agent analysing a spreadsheet. The model proposes code; a sandbox starts; the code reads data and performs calculations; the output returns to the model. A delay in starting the sandbox can extend the task even when inference capacity is available. Faster model execution cannot remove that delay by itself.

Nor does that establish that the CPU is always responsible. Storage latency, network requests, memory limits and software contention can produce similar symptoms. Buyers need to identify where requests wait before deciding which hardware to add.

Tokyo Electron’s October 30 presentation provides another industry signal: its business-environment assessment identifies growing CPU demand for preprocessing, delivering data to GPUs and peripheral control. That is management’s market assessment, not independent proof of future demand. It nevertheless describes why accelerator investment can create additional work around the accelerator rather than eliminate it.

Orders are not deployed capacity

Arm reported revenue of $1.44 billion for its second quarter of fiscal 2027, ended September 30, up 27% year over year. It also said data-center royalties doubled year over year.

Those figures establish growth in Arm’s reported business. They do not measure the severity of customers’ CPU bottlenecks or prove that a particular architecture offers the lowest operating cost.

The more consequential procurement detail is Arm’s statement that it secured CPU purchase orders from multiple customers extending through calendar 2027. First commercial Arm AGI CPU deliveries were scheduled for Q4 FYE27. Orders indicate commitments; scheduled deliveries remain forward-looking. Neither should be presented as completed deployment.

Oracle, by contrast, announced general availability of Vera CPUs on OCI. Its separate description of Rubin NVL8 as a planned configuration reinforces the distinction: a portfolio announcement can contain both available services and future products. A buyer needs the status of the specific item being purchased, not the headline’s broad implication of readiness.

Three routes, three integration boundaries

The following comparison is an architectural framework, not a finding that one route has delivered better customer outcomes.

Custom Arm-based chips put more design choices with the chip developer. Depending on the licensing arrangement, customisation can involve proprietary CPU cores or a system built around licensed cores. The developer can shape features around its workload, but must also arrange the work required to turn a design into supported hardware: verification, manufacturing, packaging, firmware and system integration.

Control is not independence. A custom chip still depends on architecture rights, fabrication capacity and a software ecosystem. The potential benefit is greater influence over product decisions. The cost is carrying more engineering and execution risk.

Licensed compute subsystems move the starting line forward. Instead of assembling every major processor building block separately, a developer starts with a more integrated design containing CPU cores and associated infrastructure. That can reduce design work, while leaving substantial responsibility for the final chip and system.

The trade-off is narrower freedom within the supplied subsystem. Does its configuration match the workload? Can the developer accept its release cadence? A shorter route to silicon is valuable only if the inherited design choices remain useful.

Finished silicon moves processor development to the supplier. Buying Arm’s own CPU would remove the buyer’s need to develop that processor. It would not remove deployment work: servers still require compatible memory, networking, firmware, operating systems and validated applications.

This route could simplify procurement and concentrate responsibility for the processor. It could also make the buyer more dependent on the supplier’s product configurations and roadmap. That is a hypothesis to examine through contracts and deployment evidence, not an established consequence of Arm’s announced orders.

Buy an operating path, not an architecture label

For enterprise buyers purchasing cloud instances or servers, these choices often sit upstream. Their immediate supplier may be a cloud provider or system vendor rather than Arm. The relevant question becomes what those upstream decisions mean for capacity, support and migration.

Start with the workload. Measure sandbox startup times, tool-service queues, memory pressure and end-to-end task completion under representative concurrency. Those are proposed evaluation criteria, not results from testing reported here.

Then ask who owns each failure. Does one supplier support the processor, firmware and server together, or must the operator coordinate several vendors? What happens when a required software package is unavailable for the chosen platform? An architecture label does not settle those questions.

Finally, examine the exit path. Moving between Arm-based systems may preserve some software investments, but common architecture does not guarantee identical peripherals, performance behaviour or operational tooling. Portability needs validation.

Who benefits? Arm can participate through licensing, royalties and finished-CPU sales. Chip developers may preserve differentiation through custom designs. Operators may benefit from less integration work—but only if the time saved exceeds the costs of migration and dependency.

The evidence supports taking CPU-side AI workloads seriously. It does not support approving any one procurement route as universally superior. The useful purchase is not simply more cores: it is sufficient capacity, a clear support boundary and a roadmap the buyer can live with.

Sources and References

• Arm: Second-quarter fiscal 2027 results, November 4, 2026—reported financial results, CPU orders and scheduled deliveries.
• Tokyo Electron: Q2 FY2027 financial presentation, October 30, 2026—management’s assessment of CPU demand around AI infrastructure.
• Oracle: Vera CPUs and Vera Rubin infrastructure on OCI, October 27, 2026—announced availability and supporting CPU workloads.