Skip to content

GitLab

You need to sign in or sign up before continuing.
Unverified Commit 97d857c0 authored by Hanming Lu's avatar Hanming Lu Committed by GitHub
Browse files

[Mamba] Increase default mamba_full_memory_ratio to 0.9 (#11679)

parent 52a54a26
Hide whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
python/sglang/srt/server_args.py
View file @ 97d857c0
...@@ -371,7 +371,7 @@ class ServerArgs: ...@@ -371,7 +371,7 @@ class ServerArgs:
# Mamba cache # Mamba cache
max_mamba_cache_size: Optional[int] = None max_mamba_cache_size: Optional[int] = None
mamba_ssm_dtype: str = "float32" mamba_ssm_dtype: str = "float32"
mamba_full_memory_ratio: float = 0.2 mamba_full_memory_ratio: float = 0.9
# Hierarchical cache # Hierarchical cache
enable_hierarchical_cache: bool = False enable_hierarchical_cache: bool = False
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment