Home / AI Cards / H200
NVIDIA

H200

Hopper · 2023 · Datacenter

Positioning

H100 with larger memory, focused on LLM inference

标签:推理大显存HBM3e

Specifications

厂商
NVIDIA
架构
Hopper
类别
Datacenter
制程
TSMC 4N
发布时间
2023
显存容量
141 GB
显存类型
HBM3e
显存带宽
4.8 TB/s
FP16 算力
989 TFLOPS
FP8 算力
1979 TFLOPS
FP4 算力
— PFLOPS
互联带宽
0.9 TB/s
功耗 TDP
700 W
状态
Mainstream

Sources

数据来自厂商官方文档与公开行业报告(NVIDIA、AMD、华为、IDC 等)。算力数值请结合精度与稀疏/稠密口径阅读;本站统一标注,避免纸面参数误导。采集时间 2026-11。

Frequently Asked Questions

Clear answers about AI accelerators, specs, precision and buying. (10 questions)

What is the memory capacity of H200?+
141 GB HBM3e with 4.8 TB/s bandwidth.
What is the FP16 throughput of H200?+
989 TFLOPS (dense).
Why is sparse compute often double the dense figure?+
Structured sparsity (e.g. 2:4) skips every other zero value in a pattern, letting hardware double useful throughput. Vendors quote sparse numbers because they look higher, but real-world gains depend on whether the model and framework exploit 2:4 sparsity. Always cite dense figures for fair comparison.
Why is memory bandwidth critical for large model inference?+
Autoregressive generation is memory-bandwidth bound: each token requires reading the entire model weights from HBM. Higher bandwidth (TB/s) directly increases tokens-per-second, often more than raw TFLOPS. This is why H100/H200/B200 emphasize HBM bandwidth.
What is the difference between GPGPU and a gaming GPU?+
A gaming GPU is tuned for raster graphics and framed around real-time rendering. A GPGPU (General-Purpose GPU) is the same silicon family but optimized and marketed for compute: larger memory, ECC support, better FP64/FP8 throughput, server cooling and virtualization. Cards like the NVIDIA A100/H100 are GPGPUs, while GeForce RTX is aimed at games.
What are Tensor Cores?+
Tensor Cores are specialized units inside NVIDIA GPUs that perform mixed-precision matrix multiply-and-accumulate in a single operation. They dramatically accelerate deep-learning math at FP16, BF16, FP8 and FP4 while using less power than CUDA cores doing the same work.
What is CoWoS and why is it a bottleneck?+
CoWoS is TSMCs 2.5D packaging technology that places a GPU die next to HBM stacks on a silicon interposer. Advanced AI chips depend on it, and limited CoWoS capacity has been a key supply constraint for NVIDIA/AMD accelerators.
What is NVLink used for?+
NVLink is NVIDIs high-speed GPU-to-GPU interconnect, allowing direct memory sharing and coherent access between GPUs at hundreds of GB/s. It is used to build multi-GPU servers (HGX/DGX) so models larger than one cards memory can be sharded efficiently.
Can a consumer GPU be used for AI training?+
Yes for small models and fine-tuning, using cards like RTX 4090. Limits include smaller VRAM (24GB), no ECC, weaker FP8/FP4 support, consumer cooling and no NVLink. For serious training, datacenter cards are more reliable and scalable.
What is FP8 and why is it popular now?+
FP8 uses 8 bits per number, halving memory and doubling throughput versus FP16 with acceptable accuracy for many LLM workloads. Hardware support in H100/B200 and frameworks like Transformer Engine made FP8 practical for both inference and training.