Skip to content

GitLab

You need to sign in or sign up before continuing.
Unverified Commit 7868cea5 authored by Lyu Han's avatar Lyu Han Committed by GitHub
Browse files

Set the default value of `max_context_token_num` 1 (#761)

parent 4bcc4f11
Hide whitespace changes
Inline Side-by-side
Showing with 3 additions and 3 deletions
docs/en/turbomind_config.md
View file @ 7868cea5
...@@ -28,7 +28,7 @@ rope_theta = 10000.0 ...@@ -28,7 +28,7 @@ rope_theta = 10000.0
size_per_head = 128 size_per_head = 128
group_size = 0 group_size = 0
max_batch_size = 64 max_batch_size = 64
max_context_token_num = 4 max_context_token_num = 1
step_length = 1 step_length = 1
cache_max_entry_count = 0.5 cache_max_entry_count = 0.5
cache_block_seq_len = 128 cache_block_seq_len = 128
......
docs/zh_cn/turbomind_config.md
View file @ 7868cea5
...@@ -28,7 +28,7 @@ rope_theta = 10000.0 ...@@ -28,7 +28,7 @@ rope_theta = 10000.0
size_per_head = 128 size_per_head = 128
group_size = 0 group_size = 0
max_batch_size = 64 max_batch_size = 64
max_context_token_num = 4 max_context_token_num = 1
step_length = 1 step_length = 1
cache_max_entry_count = 0.5 cache_max_entry_count = 0.5
cache_block_seq_len = 128 cache_block_seq_len = 128
......
lmdeploy/turbomind/deploy/target_model/base.py
View file @ 7868cea5
...@@ -48,7 +48,7 @@ class TurbomindModelConfig: ...@@ -48,7 +48,7 @@ class TurbomindModelConfig:
size_per_head: int = 128 size_per_head: int = 128
group_size: int = 0 group_size: int = 0
max_batch_size: int = 64 max_batch_size: int = 64
max_context_token_num: int = 4 max_context_token_num: int = 1
step_length: int = 1 step_length: int = 1
cache_max_entry_count: float = 0.5 cache_max_entry_count: float = 0.5
cache_block_seq_len: int = 128 cache_block_seq_len: int = 128
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment