Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
OpenDAS
vllm_cscc
Commits
2386803f
Unverified
Commit
2386803f
authored
Apr 05, 2025
by
Li, Jiang
Committed by
GitHub
Apr 04, 2025
Browse files
[CPU] Change default block_size for CPU backend (#16002)
Signed-off-by:
jiang1.li
<
jiang1.li@intel.com
>
parent
95862f7b
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
with
9 additions
and
1 deletion
+9
-1
vllm/platforms/cpu.py
vllm/platforms/cpu.py
+9
-1
No files found.
vllm/platforms/cpu.py
View file @
2386803f
...
@@ -2,6 +2,7 @@
...
@@ -2,6 +2,7 @@
import
os
import
os
import
sys
import
sys
from
importlib.util
import
find_spec
from
typing
import
TYPE_CHECKING
,
Optional
from
typing
import
TYPE_CHECKING
,
Optional
import
psutil
import
psutil
...
@@ -68,8 +69,15 @@ class CpuPlatform(Platform):
...
@@ -68,8 +69,15 @@ class CpuPlatform(Platform):
cache_config
=
vllm_config
.
cache_config
cache_config
=
vllm_config
.
cache_config
ipex_avaliable
=
find_spec
(
"intel_extension_for_pytorch"
)
is
not
None
if
cache_config
and
cache_config
.
block_size
is
None
:
if
cache_config
and
cache_config
.
block_size
is
None
:
cache_config
.
block_size
=
16
cache_config
.
block_size
=
128
if
ipex_avaliable
else
16
if
not
ipex_avaliable
and
cache_config
.
block_size
!=
16
:
raise
RuntimeError
(
f
"--block-size=
{
cache_config
.
block_size
}
requires"
" intel_extension_for_pytorch"
)
scheduler_config
=
vllm_config
.
scheduler_config
scheduler_config
=
vllm_config
.
scheduler_config
if
((
scheduler_config
.
chunked_prefill_enabled
if
((
scheduler_config
.
chunked_prefill_enabled
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment