vllm_serve.sh 151 Bytes
Newer Older
zzg_666's avatar
zzg_666 committed
1
vllm serve miromind-ai/MiroThinker-v1.5-235B --trust-remote-code --dtype float16 -tp 8 --max-model-len 32768 --gpu-memory-utilization 0.95  --port 8010