Skip to content

GitLab

Unverified Commit 7868cea5 authored by Lyu Han's avatar Lyu Han Committed by GitHub
Browse files

Set the default value of `max_context_token_num` 1 (#761)

parent 4bcc4f11
Show whitespace changes
Inline Side-by-side
Showing with 3 additions and 3 deletions
docs/en/turbomind_config.md
View file @ 7868cea5
...@@ -28,7 +28,7 @@ rope_theta = 10000.0 ...@@ -28,7 +28,7 @@ rope_theta = 10000.0
size_per_head = 128 size_per_head = 128
group_size = 0 group_size = 0
max_batch_size = 64 max_batch_size = 64
max_context_token_num = 4 max_context_token_num = 1
step_length = 1 step_length = 1
cache_max_entry_count = 0.5 cache_max_entry_count = 0.5
cache_block_seq_len = 128 cache_block_seq_len = 128
......
docs/zh_cn/turbomind_config.md
View file @ 7868cea5
...@@ -28,7 +28,7 @@ rope_theta = 10000.0 ...@@ -28,7 +28,7 @@ rope_theta = 10000.0
size_per_head = 128 size_per_head = 128
group_size = 0 group_size = 0
max_batch_size = 64 max_batch_size = 64
max_context_token_num = 4 max_context_token_num = 1
step_length = 1 step_length = 1
cache_max_entry_count = 0.5 cache_max_entry_count = 0.5
cache_block_seq_len = 128 cache_block_seq_len = 128
......
lmdeploy/turbomind/deploy/target_model/base.py
View file @ 7868cea5
...@@ -48,7 +48,7 @@ class TurbomindModelConfig: ...@@ -48,7 +48,7 @@ class TurbomindModelConfig:
size_per_head: int = 128 size_per_head: int = 128
group_size: int = 0 group_size: int = 0
max_batch_size: int = 64 max_batch_size: int = 64
max_context_token_num: int = 4 max_context_token_num: int = 1
step_length: int = 1 step_length: int = 1
cache_max_entry_count: float = 0.5 cache_max_entry_count: float = 0.5
cache_block_seq_len: int = 128 cache_block_seq_len: int = 128
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment