Commit af7b9236 authored by xuxzh1's avatar xuxzh1 🎱
Browse files

update readme

parent 7004fab5
......@@ -53,6 +53,8 @@ export HSA_OVERRIDE_GFX_VERSION=设备型号(如: gfx906对应9.0.6;gfx928
export LD_LIBRARY_PATH=/path/to/ollama/llama/build/linux-amd64/runners/rocm:$LD_LIBRARY_PATH (path/to换成到自己ollama路径)
export HIP_VISIBLE_DEVICES=所有设备号(0,1,2,3,4,5,6,...)/选择设备号
./ollama serve (选择可用设备,可通过上条命令输出结果查看)
# 新增fa和kv cache量化
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q4_0 ./ollama serve
./ollama run llama3.1
```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment