Topic

AI Infrastructure

GPU infrastructure inherited the Kubernetes control plane without inheriting any of the assumptions that make it work. Accelerators are indivisible in ways CPUs are not, ten to fifty times more expensive per unit of idle time, supply constrained on a timescale of quarters, and attached to workloads whose runtime characteristics differ so sharply between training and serving that treating them as one platform problem guarantees you will do both badly.

What gets covered here: fractional GPU strategies and their honest limits — time-slicing, MPS, MIG, and when each is the wrong answer. Multi-tenant fairness when a single job can hold a $30,000 device hostage for six days. The architectural split between training clusters, which are throughput machines tolerant of preemption, and inference fleets, which are latency machines that are not. Plus the market layer around all of this, including how AI-era naming and premium domain valuation actually behave once you look at comparable sales rather than registrar marketing.

Articles in AI Infrastructure

3 published

Other topics

Kubernetes FinOps Platform Engineering Cloud Security