Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in / Register
Toggle navigation
Menu
Open sidebar
change
sglang
Commits
001f5194
Unverified
Commit
001f5194
authored
Aug 28, 2025
by
Zhiqiang Xie
Committed by
GitHub
Aug 28, 2025
Browse files
[HiCache] change the default policy to write through (#9772)
parent
5ad296bd
Changes
2
Hide whitespace changes
Inline
Side-by-side
Showing
2 changed files
with
2 additions
and
2 deletions
+2
-2
docs/advanced_features/server_arguments.md
docs/advanced_features/server_arguments.md
+1
-1
python/sglang/srt/server_args.py
python/sglang/srt/server_args.py
+1
-1
No files found.
docs/advanced_features/server_arguments.md
View file @
001f5194
...
@@ -236,7 +236,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
...
@@ -236,7 +236,7 @@ Please consult the documentation below and [server_args.py](https://github.com/s
|
`--enable-hierarchical-cache`
| Enable hierarchical cache. | False |
|
`--enable-hierarchical-cache`
| Enable hierarchical cache. | False |
|
`--hicache-ratio`
| The ratio of the size of host KV cache memory pool to the size of device pool. | 2.0 |
|
`--hicache-ratio`
| The ratio of the size of host KV cache memory pool to the size of device pool. | 2.0 |
|
`--hicache-size`
| The size of the hierarchical cache. | 0 |
|
`--hicache-size`
| The size of the hierarchical cache. | 0 |
|
`--hicache-write-policy`
| The write policy for hierarchical cache. | write_through
_selective
|
|
`--hicache-write-policy`
| The write policy for hierarchical cache. | write_through |
|
`--hicache-io-backend`
| The IO backend for hierarchical cache. | |
|
`--hicache-io-backend`
| The IO backend for hierarchical cache. | |
|
`--hicache-storage-backend`
| The storage backend for hierarchical cache. | None |
|
`--hicache-storage-backend`
| The storage backend for hierarchical cache. | None |
...
...
python/sglang/srt/server_args.py
View file @
001f5194
...
@@ -286,7 +286,7 @@ class ServerArgs:
...
@@ -286,7 +286,7 @@ class ServerArgs:
enable_hierarchical_cache
:
bool
=
False
enable_hierarchical_cache
:
bool
=
False
hicache_ratio
:
float
=
2.0
hicache_ratio
:
float
=
2.0
hicache_size
:
int
=
0
hicache_size
:
int
=
0
hicache_write_policy
:
str
=
"write_through
_selective
"
hicache_write_policy
:
str
=
"write_through"
hicache_io_backend
:
str
=
"kernel"
hicache_io_backend
:
str
=
"kernel"
hicache_mem_layout
:
str
=
"layer_first"
hicache_mem_layout
:
str
=
"layer_first"
hicache_storage_backend
:
Optional
[
str
]
=
None
hicache_storage_backend
:
Optional
[
str
]
=
None
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment