Research · AI Infrastructure & Operations

AI Compute & Cloud Platforms

AI runs on specialized compute, usually GPUs and other accelerators, delivered through cloud platforms, specialized providers or your own data centers. Where AI runs affects cost, performance, data control and resilience.

The Workload Placement Grid

Most organizations now run AI in more than one place. The CorpExcellence Workload Placement Grid sorts each workload by two questions: how much control you need over data and where it is processed, and how large and steady its demand is. The answer gives a default home for that workload. It is a starting point for a decision, not a permanent assignment.

Managed model APIs

Pay-per-token APIs from model providers are the fastest way to start. You skip capacity planning and get new models as they ship. The trade-off is less control over data handling and model versions, plus costs that climb with usage. Use them for pilots and spiky workloads, and send calls through a gateway so you can change providers later. Check each provider’s data retention and training terms before sending sensitive content.

Private endpoints

When data must stay in a specific region or tenancy but demand is uneven, deploy models inside your own cloud account. That can mean provider-hosted models with regional and private networking options, or open-weight models on managed serving. You keep data boundaries and audit logs while still paying mostly for what you use. This is often the right home for regulated data in early production.

Reserved capacity

Steady, high-volume inference or regular training runs justify committed capacity from a hyperscaler or a specialist GPU cloud. Commitments lower unit prices but lock in spend, so size them against measured production usage, not forecasts made during a pilot. Specialist GPU clouds can offer faster access to new accelerators, but review their security attestations and exit terms as carefully as a hyperscaler’s.

Owned or colocated

Sustained workloads with strict sovereignty or latency needs can justify your own accelerators in a data center or colocation facility. Check power density and cooling first, since many facilities were not built for dense GPU racks. Budget for the specialists who will run the cluster, not only the hardware. Accelerator generations turn over quickly, so plan the refresh cycle before you buy.

Choosing between hyperscalers, neoclouds, sovereign cloud and private AI?

AI Infrastructure Sourcing 2026 turns this placement grid into a full decision method: workload tiers, hard gates, a weighted decision matrix and a 50-question provider due-diligence questionnaire.

Get the full report →

Where the industry is now

Spending is shifting from training toward inference as companies put AI into production, and server makers now sell inference-optimized systems for on-premises use (Computerworld, CES 2026). Accelerator choice is also widening: Google Cloud made its Ironwood TPU generally available in March 2026, alongside the GPU fleets every major cloud offers. On the software side, Kubernetes made Dynamic Resource Allocation generally available in version 1.34, giving clusters a standard way to request GPUs and other devices, and the Gateway API Inference Extension adds model-aware routing for self-hosted models. Together these make a portable, multi-venue setup more practical than it was two years ago.

Common pitfalls

• Buying reserved or owned capacity before you have production usage data.
• Hard-coding one provider’s SDK into every application.
• Ignoring egress and data movement charges when splitting work across venues.
• Treating placement as a one-time decision instead of a recurring review.