Almost every AI model you use, from ChatGPT and Claude to DeepSeek and Kimi, runs on a handful of NVIDIA data center GPUs. In 2026 there are five generations in active use: H100, H200, B200, B300 and the brand-new Vera Rubin. Their names appear in every AI announcement and every cloud price list, but the differences are not obvious. This guide compares them in plain English, with the specs that matter for AI (memory, bandwidth and low-precision compute), what each costs to rent today, and which one makes sense for inference, fine-tuning and training.
- H100 (Hopper, 2022): 80 GB HBM3, 3.35 TB/s. Still the workhorse; cheapest to rent at about $2-$4/hr.
- H200 (Hopper, 2024): same chip, 141 GB HBM3e at 4.8 TB/s. Better for big-model inference.
- B200 (Blackwell, 2025): 180-192 GB HBM3e, 8 TB/s, native FP4. Roughly 2-3x an H100 for inference.
- B300 (Blackwell Ultra): 288 GB HBM3e, about 15 PFLOPS dense FP4. Built for reasoning models with long context.
- Vera Rubin (2026): 288 GB HBM4 at up to 22 TB/s, 50 PFLOPS FP4 inference. Shipping to the biggest clouds in H2 2026; broad rental access mostly in 2027.
- For most people: rent an H100 for fine-tuning and testing; H200 or B200 for serving 100B+ models.
Spec Comparison Table
Per-GPU figures for the SXM versions (the ones used in 8-GPU servers and NVL72 racks). Compute is dense, without sparsity.
| Spec | H100 | H200 | B200 | B300 | Vera Rubin |
|---|---|---|---|---|---|
| Architecture | Hopper | Hopper | Blackwell | Blackwell Ultra | Rubin |
| Memory | 80 GB HBM3 | 141 GB HBM3e | 180-192 GB HBM3e | 288 GB HBM3e | 288 GB HBM4 |
| Bandwidth | 3.35 TB/s | 4.8 TB/s | 8 TB/s | 8 TB/s | up to 22 TB/s |
| FP8 (dense) | about 1.98 PFLOPS | about 1.98 PFLOPS | 4.5 PFLOPS | about 4.5 PFLOPS | about 17.5 PFLOPS (estimate) |
| FP4 (dense) | not supported | not supported | 9 PFLOPS | about 15 PFLOPS | 50 PFLOPS inference / 35 training |
| Power (TDP) | 700 W | 700 W | about 1,000-1,200 W | about 1,400 W | about 2,300 W (max) |
| NVLink per GPU | 900 GB/s (Gen 4) | 900 GB/s (Gen 4) | 1.8 TB/s (Gen 5) | 1.8 TB/s (Gen 5) | 3.6 TB/s (Gen 6) |
Figures come from NVIDIA data as compiled by Spheron (updated 26 October 2026) and Thunder Compute (updated 4 November 2026). Rubin numbers are NVIDIA’s preliminary figures and may change; its FP8 value is derived, not officially confirmed. B200 is listed as 192 GB of physical HBM, of which about 180 GB is usable in most systems.
What the Specs Mean for AI
- Memory capacity decides how big a model fits on one GPU or one server. An 8x H100 server has 640 GB; an 8x B300 server has 2.3 TB. Trillion-parameter models need the latter, or need to be split over several servers.
- Bandwidth decides how fast tokens come out when generating. It is the main reason the H200 beats the H100 at inference even though the chip is the same.
- FP8 and FP4 compute decide training speed and how many users you can serve at once. Blackwell added native FP4 (NVFP4), which roughly doubles throughput again for models that are quantized to 4 bits.
- NVLink connects GPUs. In an NVL72 rack, 72 GPUs act like one giant GPU, which is how the largest models are served.
The same rules apply on a home PC, just at smaller scale. My VRAM guide explains how model size, quantization and context map to memory.
H100: The Workhorse
The H100 launched in 2022 and powered the first wave of large AI models. With 80 GB of HBM3 and FP8 support through its Transformer Engine, it is still the most widely available data center GPU in 2026, which makes it the cheapest to rent. Rental prices have fallen a lot: RunPod lists an H100 SXM from $2.69 (Community) to $3.99 per hour, and Vast.ai verified hosts list it around $1.80-$2.50.
Best for: fine-tuning and LoRA training up to about 70B with QLoRA, serving 8B-70B models, research and experiments.
H200: More Memory, Same Chip
The H200 is an H100 with bigger, faster memory: 141 GB of HBM3e at 4.8 TB/s. The compute is the same, but because LLM inference is limited by memory bandwidth, it generates tokens noticeably faster and fits bigger models or longer contexts per GPU. A 70B model at FP8 (about 70 GB) fits on a single H200 with room for a large KV cache.
Best for: serving 70B-200B models, long-context inference, anyone who keeps running out of memory on H100.
B200: Blackwell and FP4
Blackwell is a big jump. The B200 is two dies in one package, with 180-192 GB of HBM3e at 8 TB/s and native FP4. At FP8 it offers about 2.3x the dense compute of an H100, and FP4 roughly doubles that again for inference. It also introduced the GB200 NVL72 rack, which links 72 GPUs with NVLink 5. Lambda rents B200s at $6.69 per GPU-hour on demand; RunPod lists them at $7.99.
Best for: large-scale inference of frontier-size MoE models, training mid-size models quickly.
B300: Blackwell Ultra
The B300 (Blackwell Ultra) increases memory to 288 GB of HBM3e per GPU and boosts FP4 compute to about 15 PFLOPS dense, at around 1,400 W. NVIDIA designed it for reasoning models, which generate many more tokens per answer and need long contexts. It ships in HGX B300 servers and GB300 NVL72 racks (about 140 kW per rack). RunPod lists the B300 at $8.99 per hour.
Best for: serving reasoning and agent models at scale, trillion-parameter MoE models with long context.
Vera Rubin: HBM4 and NVLink 6
Rubin is NVIDIA’s 2026 architecture. Each Rubin GPU carries 288 GB of HBM4 at up to 22 TB/s, almost three times the bandwidth of B300, and NVIDIA rates it at 50 PFLOPS of NVFP4 for inference. It pairs with NVIDIA’s new Vera CPU. The flagship Vera Rubin NVL72 rack combines 72 Rubin GPUs and 36 Vera CPUs, with 20.7 TB of HBM4, 260 TB/s of NVLink 6 bandwidth and 3,600 PFLOPS of NVFP4 inference, fully liquid-cooled at roughly 190-230 kW.
According to Thunder Compute, Rubin entered full production in January 2026 and is shipping to hyperscalers and neoclouds in the second half of 2026. Spheron lists the first cloud cohort as AWS, Google Cloud, Microsoft Azure, Oracle, CoreWeave, Lambda, Nebius and Nscale, with broader availability expected in 2027. Spheron estimates launch rental prices of $15-$25 per hour at hyperscalers, but that is an estimate; official pricing has not been announced.
All that HBM4 is a big reason consumer memory is so expensive this year. See why RAM is so expensive in 2026.
Rental Prices in November 2026
| GPU | RunPod (per GPU-hour) | Lambda | Vast.ai (typical) |
|---|---|---|---|
| H100 PCIe | $2.89 (Community from $1.99) | – | $0.90-$2.27 |
| H100 SXM | $3.99 (Community from $2.69) | $3.99 | $1.80-$2.50 |
| H200 | $5.29 (Community from $3.59) | – | $3.50-$5.50 |
| B200 | $7.99 | $6.69 | $6-$12 (rare) |
| B300 | $8.99 | – | – |
RunPod and Lambda prices are from their pricing pages on 5 November 2026. More providers, consumer cards and hidden costs are in my cheapest cloud GPU guide.
Which One Should You Use?
| Your job | Best choice | Why |
|---|---|---|
| LoRA / QLoRA fine-tuning up to 70B | 1x H100 | Cheapest per hour, mature software |
| Serving a 70B model to a few users | 1x H200 | Whole model at FP8 on one GPU |
| Serving 200B-1T MoE models | 8x H200 or 8x B200 | Total memory and FP4 throughput |
| Reasoning models with very long context | B300 | 288 GB per GPU |
| Pre-training at scale | B200/B300 clusters now, Rubin when available | Compute per watt |
| Running a 32B model for yourself | None of these | An RTX 5090 or PRO 6000 is far cheaper |
For single-user local work, a data center GPU is usually overkill. A rented RTX 5090 or RTX PRO 6000 costs a fraction of an H100. See the best workstation GPUs for AI.
What About AMD Instinct?
AMD’s Instinct MI355X matches the B300 on memory with 288 GB of HBM3e at 8 TB/s and is offered by several clouds, often at lower prices than NVIDIA. AMD’s next platform, Helios, is a rack-scale system built around the MI455X to compete with Vera Rubin NVL72. Software (ROCm) has improved a lot, and vLLM and SGLang support Instinct well, but CUDA remains the default for most training code.
What an 8-GPU Node Costs to Rent
Large models are usually served or trained on 8-GPU nodes. Using the RunPod on-demand prices above, here is what a full node costs if it runs around the clock:
| Node | Per hour | Per day (24 h) | Per month (30 days) | Total GPU memory |
|---|---|---|---|---|
| 8x H100 SXM | $31.92 | $766 | About $22,980 | 640 GB |
| 8x H200 | $42.32 | $1,016 | About $30,470 | 1,128 GB |
| 8x B200 | $63.92 | $1,534 | About $46,020 | 1,440 GB |
| 8x B300 | $71.92 | $1,726 | About $51,780 | 2,304 GB |
These are on-demand rates. Reserved contracts, community clouds and spot capacity are often much cheaper, while big clouds such as AWS, Azure and Google Cloud usually charge more per GPU-hour. The newer cards cost more per hour but can be cheaper per token served, because a B200 or B300 node can handle far more requests at FP4 or FP8.
How to Rent Your First Data Center GPU
- Size the job: add up the model weights (parameters x bytes per weight) plus KV cache, and pick the smallest GPU that fits. My VRAM guide has the formulas.
- Pick a provider and template: choose a PyTorch or vLLM template so CUDA and drivers are already set up.
- Attach storage for model weights so you do not pay to download them again every time.
- Test small first: run a short job on one GPU before you scale to eight.
- Stop or delete the pod as soon as you finish. Idle GPUs bill by the minute or hour.
A full step-by-step walkthrough is in my cheapest cloud GPU guide.
Software Support: What Runs Where
| Feature | H100 / H200 (Hopper) | B200 / B300 (Blackwell) |
|---|---|---|
| FP8 | Yes | Yes |
| FP4 (NVFP4) | No | Yes, the big speed-up for inference |
| vLLM, SGLang, TensorRT-LLM | Mature | Supported; use recent versions and CUDA 12.8 or newer |
| Older research code | Usually works | May need rebuilding for the newer architecture |
Frequently Asked Questions
What is the difference between H100 and H200?
They use the same Hopper chip. The H200 has 141 GB of HBM3e at 4.8 TB/s instead of 80 GB of HBM3 at 3.35 TB/s, so it fits larger models and generates tokens faster.
Is B200 better than H100?
Yes. The B200 has more than twice the memory, about 2.4x the bandwidth, about 2.3x the dense FP8 compute and native FP4. It also costs about twice as much per hour to rent.
What is the difference between B200 and B300?
B300 (Blackwell Ultra) has 288 GB of HBM3e instead of 180-192 GB and about 15 PFLOPS of dense FP4 instead of 9, at higher power. It targets reasoning models and long contexts.
When can I rent NVIDIA Rubin?
Rubin is shipping to the largest clouds and a few neoclouds in the second half of 2026. Wider rental availability is expected in 2027.
How much does an H100 cost to rent?
Roughly $1.80-$4 per hour on specialist clouds in November 2026, and $7-$12 per hour on-demand at AWS, Google Cloud and Azure.
How much does it cost to rent 8 H100s for a day?
About $766 at RunPod’s on-demand H100 SXM price of $3.99 per GPU-hour in November 2026. Community and reserved pricing can be much lower.
Final Thoughts
Think of the five generations as steps in memory and bandwidth: 80 GB, 141 GB, 180-192 GB, 288 GB, then 288 GB of much faster HBM4. For anything you can do on one GPU, the H100 is the cheapest and most available. For big models, H200 and B200 give the best value right now, and Rubin will reset the top end in 2027. If you want to run models at home instead, start with the best GPU for local LLMs.
