Positioning
Amazon in-house training ASIC, tightly integrated with AWS
标签:
Specifications
厂商
AWS
架构
NeuronCore-v2
类别
ASIC
制程
TSMC 4nm
发布时间
2024
显存容量
96 GB
显存类型
HBM3e
显存带宽
— TB/s
FP16 算力
— TFLOPS
FP8 算力
— TFLOPS
FP4 算力
— PFLOPS
互联带宽
0 TB/s
功耗 TDP
— W
状态
Current
Sources
数据来自厂商官方文档与公开行业报告(NVIDIA、AMD、华为、IDC 等)。算力数值请结合精度与稀疏/稠密口径阅读;本站统一标注,避免纸面参数误导。采集时间 2026-11。
Frequently Asked Questions
Clear answers about AI accelerators, specs, precision and buying. (10 questions)
What is the memory capacity of Trainium2?+
96 GB HBM3e with — TB/s bandwidth.
What is the FP16 throughput of Trainium2?+
Not disclosed TFLOPS (dense).
What is AWS Trainium2 used for?+
Trainium2 is AWSs custom training ASIC offered in EC2 instances, designed to cut cost for large-model training within the AWS ecosystem. It is tightly integrated with AWS tooling but not generally available as a standalone card.
What is the difference between inference and training ASICs?+
Training ASICs emphasize high precision throughput, large memory and scalability; inference ASICs prioritize low latency, low precision (INT8/FP8) and cost per token. Examples: AWS Trainium (training) vs Inferentia (inference).
What is the difference between a GPU and an NPU?+
A GPU (Graphics Processing Unit) is a general-purpose parallel processor with many cores, excellent at matrix math and flexible enough for training and rendering. An NPU (Neural Processing Unit) is a specialized chip designed only for neural-network inference and training, with dataflow architectures that trade flexibility for higher efficiency. In short: GPU is general-purpose and programmable; NPU is domain-specific and more power-efficient for fixed AI workloads.
What is an AI accelerator card?+
An AI accelerator card is a PCIe or OAM add-in card that offloads machine-learning training and inference from the CPU. It typically carries a GPU, NPU or ASIC plus high-bandwidth memory (HBM) and is used in servers and workstations to run large neural networks efficiently.
What is the difference between training and inference hardware needs?+
Training needs very high compute throughput, large memory capacity and fast interconnect to scale across many GPUs. Inference prioritizes low latency, high throughput per watt, lower precision (INT8/FP8/FP4) and cost. Training cards are usually top-tier (H100/B200); inference can use mid-range or NPU/ASIC solutions.
What is HBM and how is it different from GDDR?+
HBM (High Bandwidth Memory) stacks DRAM vertically and connects via a wide 1024/2048-bit interposer, delivering far higher bandwidth and lower power than GDDR, which uses wide-but-shorter parallel buses on the PCB. AI accelerators use HBM3/HBM3e/HBM4 because large models are memory-bandwidth bound.
What does OAM mean in AI accelerator modules?+
OAM (OCP Accelerator Module) is an open hardware standard from the Open Compute Project defining the module form factor, cooling and high-speed interfaces for AI accelerators. It lets different vendors cards fit the same server baseboard, easing system integration.
Is more VRAM always better?+
Not always. Larger VRAM lets you hold bigger models and batches, but if the workload is compute-bound, extra memory adds cost without speed gains. Bandwidth often matters more than raw capacity. Match VRAM to model size plus KV-cache headroom.
No matching questions.