Hardware: choose by the bottleneck

A GPU specification is a set of constraints, not a speed guarantee. Size memory, estimate data movement, then measure the intended workload.

Will it fit?

Account for weights, cache, activations, runtime workspaces, and headroom. Use bytes and units before choosing a device.

Compare arithmetic, HBM bandwidth, capacity, and supported low-precision kernels. Profiles decide which specification matters.

Draw the real node/network topology. Include collective and KV-transfer costs in the estimate.

Compare full replicas, hardware partitions, adapters, and local CPU/unified-memory execution for the actual workload.