- An AMD GPU capable of supporting ROCm and an appropriate amdgpu driver
- Assumes your ROCm tools are installed in /opt/rocm/ and rocminfo is in your path (often found in /opt/rocm/bin/), if not edit the Makefile to match your distro and/or update your PATH before running.
Linux distribution (Ubuntu, MacOS, etc.) + CUDA >= 10.0. LLM.int8() requires Turing or Ampere GPUs.
- Assumes your ROCm tools are installed in /opt/dtk/