Everything on Precision: Numeric Formats in Modern ML and HPC
A walkthrough of numeric formats that matter today (FP32, TF32, FP16, BF16, FP8, FP4, INT8, INT4), how they behave, when to use each, and where the pitfalls hide.
A walkthrough of numeric formats that matter today (FP32, TF32, FP16, BF16, FP8, FP4, INT8, INT4), how they behave, when to use each, and where the pitfalls hide.