Skip to content

GitLab

Commit af7b9236 authored by xuxzh1's avatar xuxzh1 🎱
Browse files

update readme

parent 7004fab5
Show whitespace changes
Inline Side-by-side
Showing with 2 additions and 0 deletions
README.md
View file @ af7b9236
...@@ -53,6 +53,8 @@ export HSA_OVERRIDE_GFX_VERSION=设备型号（如: gfx906对应9.0.6；gfx928 ...@@ -53,6 +53,8 @@ export HSA_OVERRIDE_GFX_VERSION=设备型号（如: gfx906对应9.0.6；gfx928
export LD_LIBRARY_PATH=/path/to/ollama/llama/build/linux-amd64/runners/rocm:$LD_LIBRARY_PATH (path/to换成到自己ollama路径) export LD_LIBRARY_PATH=/path/to/ollama/llama/build/linux-amd64/runners/rocm:$LD_LIBRARY_PATH (path/to换成到自己ollama路径)
export HIP_VISIBLE_DEVICES=所有设备号（0,1,2,3,4,5,6,...）/选择设备号 export HIP_VISIBLE_DEVICES=所有设备号（0,1,2,3,4,5,6,...）/选择设备号
./ollama serve （选择可用设备，可通过上条命令输出结果查看） ./ollama serve （选择可用设备，可通过上条命令输出结果查看）
# 新增fa和kv cache量化
OLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q4_0 ./ollama serve
./ollama run llama3.1 ./ollama run llama3.1
``` ```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment