"...git@developer.sourcefind.cn:2222/OpenDAS/vllm_cscc.git" did not exist on "ecca3fee761c9dd710daf3acb2e646d10fb631d7"
Unverified Commit 3a886bd5 authored by Reid's avatar Reid Committed by GitHub
Browse files

[Misc] small improve (#18680)


Signed-off-by: default avatarreidliu41 <reid201711@gmail.com>
Co-authored-by: default avatarreidliu41 <reid201711@gmail.com>
parent 35be8fad
......@@ -15,7 +15,7 @@ pip install bitsandbytes>=0.45.3
vLLM reads the model's config file and supports both in-flight quantization and pre-quantized checkpoint.
You can find bitsandbytes quantized models on <https://huggingface.co/models?search=bitsandbytes>.
You can find bitsandbytes quantized models on [Hugging Face](https://huggingface.co/models?search=bitsandbytes).
And usually, these repositories have a config.json file that includes a quantization_config section.
## Read quantized checkpoint
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment