benchmarking/inference_benchmark.py · b1f80b8acc6f8cfa7932dece460f6b600466dd34 · OpenDAS / bitsandbytes

[CUDA] Branchless NF4/FP4 kDequantizeBlockwise kernel for faster dequantization (#1746) · b1f80b8a

Mohamed Hisham authored Sep 18, 2025

* Added branchless LUT-based dequantization for FP4 and NF4

* Added extra command line options to control reproducibility

* Restore FP4 quantization/dequantization order

b1f80b8a

inference_benchmark.py 5.84 KB

Replace inference_benchmark.py