You need to sign in or sign up before continuing.
Unverified Commit 36c28688 authored by Merve Noyan's avatar Merve Noyan Committed by GitHub
Browse files

Added note on weight-cache-override (#994)



Added note on serving supported models from a different folder without
re-downloading them.

---------
Co-authored-by: default avatarNicolas Patry <patry.nicolas@protonmail.com>
parent a0498642
...@@ -28,6 +28,12 @@ AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")` ...@@ -28,6 +28,12 @@ AutoModelForCausalLM.from_pretrained(<model>, device_map="auto")`
AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto") AutoModelForSeq2SeqLM.from_pretrained(<model>, device_map="auto")
``` ```
If you wish to serve a supported model that already exists on a local folder, just point to the local folder.
```bash
text-generation-launcher --model-id <PATH-TO-LOCAL-BLOOM>
``````
## Supported Hardware ## Supported Hardware
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment