"CONTRIBUTING.md" did not exist on "db2b0b7925739341002fe6ac6d748fcbf3ec33bf"
Disable CUDA peer access as a workaround for multi-gpu inference bug (#1261)
When CUDA peer access is enabled, multi-gpu inference will produce garbage output. This is a known bug of llama.cpp (or nvidia). Until the upstream bug is fixed, we can disable CUDA peer access temporarily to ensure correct output. See #961.
Showing
Please register or sign in to comment