On-Demand vs Reserved GPU Instances: Total Cost of Ownership
Cloud GPU pricing comes in two fundamental models: on-demand (pay by the hour, no commitment) and reserved (commit to usage, get a discount). Choosing the wrong model for your workload pattern can double your annual GPU bill. This guide breaks down the real total cost of ownership for both models and provides a decision framework based on workload characteristics.
How the Pricing Models Work
On-Demand Instances
You pay a fixed hourly rate for each GPU. Start and stop at any time. No upfront payment, no minimum commitment, no termination penalty. Billing is per-second or per-minute with a minimum of one minute.
Typical on-demand rates for common cloud GPUs in 2026:
| GPU | On-Demand Rate (per hour) | Monthly (730 hours) |
|---|---|---|
| RTX 3090 | $0.15 | $109.50 |
| RTX 4090 | $0.35 | $255.50 |
| A4000 | $0.20 | $146.00 |
| A10 | $0.45 | $328.50 |
| A100 80GB | $1.20 | $876.00 |
| H100 | $2.50 | $1,825.00 |
On-demand is optimal when your GPU usage is intermittent, unpredictable, or still in the experimentation phase.
Reserved Instances
You commit to a minimum usage term (typically 1 month, 6 months, or 1 year) in exchange for a discounted hourly rate. Discounts range from 20% to 60% depending on term length and payment schedule.
Common reservation models:
- Monthly reserved. 20–30% discount. Pay at the start of each month. Can change or cancel at month end.
- Annual reserved (monthly pay). 35–45% discount. Commit to 12 months, pay monthly. Early termination fees apply.
- Annual reserved (all upfront). 45–60% discount. Pay the full year upfront. No refunds for early termination.
Reserved instances make sense when you have predictable, steady-state GPU workloads — for example, a production inference endpoint running 24/7 or a recurring weekly training job.
The Break-Even Calculation
The fundamental question: how many hours per month must you run a GPU before reserving beats on-demand?
For a reserved instance with a D% discount and term T months:
Break-even hours/month = (1 - D) × 730
At 30% discount (monthly reserved): 0.70 × 730 = 511 hours/month. If you run a GPU more than 511 hours (about 70% of the month), reserved pricing wins.
At 50% discount (annual upfront): 0.50 × 730 = 365 hours/month. Reserved wins at just 50% utilization.
But the real-world calculation is more nuanced because on-demand workloads often have hidden costs.
Hidden Costs That Change the Math
Storage Costs During Idle Periods
When you stop an on-demand GPU instance but keep the attached storage, you continue paying for that storage. If your workflow is "train for 8 hours, analyze results for 2 days, train again," you are paying for 48 hours of idle storage between runs.
A 500 GB NVMe volume at $0.10/GB/month costs $50/month whether the GPU is running or not. With S3-compatible object storage at $2.49/TB/month, the same 500 GB costs $1.25/month — a 40× difference. Architecting checkpoints and datasets to live on cheap object storage and staging them to local NVMe only during active runs dramatically reduces idle costs.
Data Transfer Costs
Some providers charge for data egress — moving data out of their cloud. At $0.05–$0.12/GB, downloading a 100 GB dataset costs $5–$12 each time. If you process multiple datasets or download results regularly, transfer costs can exceed compute costs.
BHK Cloud does not charge for data egress, which eliminates this variable from the TCO equation.
Instance Availability Risk
Reserved instances guarantee capacity. During GPU shortages (which became common in 2025–2026), on-demand users may face "insufficient capacity" errors when trying to launch popular GPU types. A reserved instance is your insurance against spot-market volatility.
Setup and Teardown Overhead
On-demand users often script environment setup (installing dependencies, mounting datasets, configuring networking) because they spin up fresh instances. Reserved users maintain persistent environments that are always ready. The 15–30 minutes per session spent on setup adds up: at 20 sessions per month, that is 5–10 hours of non-GPU engineering time.
Workload Patterns and Recommended Models
Continuous Production Inference
An API serving LLM responses 24/7 with consistent traffic. Recommendation: annual reserved, all upfront. The GPU runs 730 hours/month — well past any break-even. The 50%+ discount is free money.
Periodic Weekly Training
Training runs every Monday for 12 hours. That is roughly 52 hours/month. Recommendation: on-demand. At 52 hours/month, no reservation discount beats on-demand pricing. Use spot/preemptible instances for additional savings during training.
Batch Inference — End of Month
Processing a month's worth of data in a 3-day burst (72 hours). Recommendation: on-demand. The bursty pattern never reaches break-even. Spin up multiple GPUs in parallel to finish faster, then shut down.
Research and Experimentation
Unpredictable usage: some weeks 80 hours, some weeks 0 hours. Recommendation: on-demand. The variability makes commitment risky. Start with on-demand; if a pattern emerges after 3 months, re-evaluate.
Steady Training Pipeline
Training a model continuously with periodic fine-tuning — roughly 500 GPU-hours per month. Recommendation: monthly reserved. At 500 hours/month, you cross the break-even for a 30% monthly discount but fall short of the annual commitment threshold unless the pipeline is reliably steady.
Decision Framework
Answer these four questions to choose the right model:
- How many GPU-hours per month do you consistently use? Below 365 hours/month → on-demand is likely cheaper. Above 511 hours/month → reserved always wins.
- How predictable is your usage? If you can forecast with 80%+ confidence for 6–12 months, annual reserved is safe. If not, start with monthly reserved or on-demand.
- Do you need guaranteed capacity? During GPU shortages, reserved instances are the only way to guarantee access to popular GPU types.
- What is your tolerance for management overhead? Reserved instances reduce operational toil (persistent environments, no cold starts). Value your engineering time.
Hybrid Strategy: The Best of Both
Most teams benefit from a hybrid approach:
- Reserved baseline. Reserve enough GPU capacity to cover your minimum steady-state workload (e.g., production inference + recurring training).
- On-demand burst. Use on-demand or spot instances for burst capacity: extra training runs, batch jobs, experimentation.
- Cold storage. Keep datasets and checkpoints on S3-compatible object storage ($2.49/TB/month). Stage to NVMe only during active runs. Never pay idle SSD costs.
Frequently Asked Questions
Can I change GPU types with a reservation?
Most providers let you exchange reserved instances within the same GPU family (e.g., one A100 for another A100 in a different region). Cross-family exchanges (A100 → H100) typically require buying a new reservation. Check provider policies before committing.
What happens if I stop using a reserved instance mid-term?
Monthly reservations: you pay for the current month, then the reservation ends. Annual reservations: early termination fees typically equal 30–50% of the remaining commitment. Read the termination clause before signing annual contracts.
Are spot instances cheaper than reserved?
Spot instances are 60–90% cheaper than on-demand but can be terminated with 30–120 seconds notice. For fault-tolerant workloads with checkpointing (training, batch jobs), spot pricing beats reserved. For stateful or latency-sensitive workloads (inference, interactive development), reserved is safer.
How do I track GPU utilization to make this decision?
Most cloud GPU providers include dashboards showing GPU-hours per month. If yours does not, instrument your instances: log start/stop times and aggregate monthly. Three months of data is usually enough to identify a pattern.
BHK Cloud offers on-demand RTX 3090 instances at $0.15/hr with no egress fees and S3-compatible storage at $2.49/TB/month. No commitments, no hidden costs. Rent a GPU or explore storage.