In 2019 my assignment at MTS Cloud was to turn GPUs into a public cloud product. We had NVIDIA V100 cards, we had virtualization with vGPU profiles, and we had clients who wanted to train models without buying hardware. What we did not have was a product: nobody had decided what exactly we sell, at what price, at what availability, and who answers when a node dies at 3 a.m.
Getting there took most of 2019 and 2020. The engineering was rarely the hard part. The hard part was economic: every configuration is a price, every promise is a cost, and utilization decides whether the whole line makes money. Seven years later, cost-per-token is the tariff of the AI era, and I keep recognizing the same problems in new clothes.
The unit is not the GPU — it is the usable compute-hour
A V100 rented as a dedicated instance and the same card sliced into vGPU profiles are different products with different economics. Slicing raises the potential revenue per card: eight clients can share what one would have hogged. It also raises fragmentation: a card holding two half-empty profiles cannot host a full one, and every profile boundary is a small bet on what demand will look like tomorrow.
Meanwhile idle GPU capital burns at exactly the same rate whether booked or not. Depreciation, power, space and financing do not pause while the scheduler waits for a tenant. So the tariff has to be designed against the utilization curve you can actually reach — not the one on the vendor slide. That is why commitment discounts exist on both sides of the market: a reserved-instance discount is a hedge where the client trades flexibility for price and the operator trades margin for predictability. Neither side should pretend it is a gift.
The curve also has a shape, and pretending otherwise wastes money. Training demand arrives in waves — nights, weekends, end of quarter — while inference demand is a flat line with spikes. A tariff can reshape that: off-peak pricing moves tolerant jobs into the valleys, quotas protect the steady tenants from the bursty ones. But a tariff that reshapes demand is a product decision with engineering consequences, agreed with the scheduler, the capacity plan and support — not a number improvised for a deal.
Configurations are promises
A configuration is not a line in a catalog; it is a promise about isolation and performance. A vGPU profile is nominally a fraction of the card, but the client experiences it as a fraction of the machine: memory bandwidth, driver behavior, noisy neighbors. When the promise and the experience diverge, the difference arrives as a support ticket and, eventually, as churn.
Two client patterns dominated. Training was bursty: big allocations, runs lasting days, restarts tolerable if checkpointing was disciplined. Inference was steady: small, latency-sensitive, always-on. Almost everything else was a mixture. Right-sizing mattered more than catalog breadth — a mis-sized configuration produced the worst of both worlds, an unhappy client and a poor margin. And every extra SKU multiplied the support surface: documentation, monitoring, capacity planning. The minimal set that covers the dominant workloads beats a long menu, every time.
GPU workloads skew, too. Some are memory-hungry and compute-idle; others pack arithmetic and leave bandwidth asleep. Counting only fractions of a card hides this, which is why the honest catalog describes memory and compute separately and the honest operator sizes profiles against what clients actually run, not against what divides cleanly.
Training and inference are different products on one silicon
They ride the same hardware and share nothing else. Training wants peaks and large contiguous allocations; inference wants density, steadiness and low tail latency. A maintenance window is a rounding error for a checkpointed training job and an outage for an inference endpoint. The mix sitting on a node at any moment is the margin — a fleet that is all training on Monday and all inference on Friday is paying for two fleets and using one.
We treated it as one fleet with two products on top, and the catalog decisions followed from that: which configurations serve which duty cycle, what gets scheduled where, what the tenant sees when their workload class changes. The same physics is why today’s inference providers obsess over batching and queueing: the silicon does not care about your product taxonomy, only your scheduler does.
Availability is priced, not assumed
When a GPU node fails, every tenant on it feels the failure at once; with vGPU that is by design, one card holding many promises. Live migration for GPU workloads was limited, so maintenance became a product event that needed scheduling, communication and a story — not an ops footnote. Spare capacity for failures is not idle waste; it is inventory, and it belongs in the tariff.
The general rule I took away: support cost is cost of goods sold. The availability number in a service description is not marketing copy — it is the amortized price of spare nodes, monitoring, failover paths and the people who answer at 3 a.m. An availability promise you have not costed is a liability you have already signed.
What this maps to in 2026
Cost-per-token is the tariff of the inference era, and it is decided by the same three levers as our 2019 tariff: what the unit of delivered work is, what utilization the operator can reach, and what promise attaches to it. Batch sizes, quantization and KV-cache management are the new vGPU profiles — ways to slice one piece of silicon into sellable fractions. The vocabulary changed; the bin-packing did not.
Commitments came back into my life from the other side. In FinOps work at CloudLinux I now run the discipline I once faced as a seller: coverage checks before AWS commitments, a hard separation between what a projection says and what the bill measured. The 2019 lesson holds with interest — never confuse the forecast with the invoice.
And agents changed the duty cycle again. Agent fleets are spiky, parallel and long-tail; they look like 2019 training bursts, except they run on inference silicon and are triggered by software that retries without patience. Whoever prices agent compute this decade is answering the 2019 question one more time: what is the unit, what is the promise, and what does idle cost.
The boundary
This was a historical product, at one company, in one market. Its internal numbers stay internal, and the forecasts we made publicly at AI Journey 2020 were forecasts — not achieved metrics, then or now. What transferred is not a number but a discipline: price the unit of delivered work, watch utilization before capacity, and treat every configuration as a promise with a cost attached.
If you are building AI infrastructure today: the silicon is faster and the market is larger, but the questions have not changed. That is, incidentally, why I find them comfortable.