Skip to content

GitLab

Unverified Commit 5b9e454a authored by Lyu Han's avatar Lyu Han Committed by GitHub
Browse files

fix local kv head num (#806)

parent bd7c4e39
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
src/turbomind/models/llama/LlamaV2.cc
View file @ 5b9e454a
...@@ -76,7 +76,7 @@ LlamaV2<T>::LlamaV2(size_t head_num, ...@@ -76,7 +76,7 @@ LlamaV2<T>::LlamaV2(size_t head_num,
end_id_(end_id), end_id_(end_id),
hidden_units_(head_num * size_per_head), hidden_units_(head_num * size_per_head),
local_head_num_(head_num / tensor_para.world_size_), local_head_num_(head_num / tensor_para.world_size_),
local_kv_head_num_(head_num / tensor_para.world_size_), local_kv_head_num_(kv_head_num / tensor_para.world_size_),
weights_(weights), weights_(weights),
tensor_para_(tensor_para), tensor_para_(tensor_para),
stream_(stream), stream_(stream),
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment