"docs/source/vscode:/vscode.git/clone" did not exist on "6178897d3ceb6967de3c03ea04e0d902ebdb3f0a"
Disable CUDA peer access as a workaround for multi-gpu inference bug (#1261)
When CUDA peer access is enabled, multi-gpu inference will produce garbage output. This is a known bug of llama.cpp (or nvidia). Until the upstream bug is fixed, we can disable CUDA peer access temporarily to ensure correct output. See #961.
Showing
Please register or sign in to comment