examples/tensorize_vllm_model.py · 51602eefd38250325e541abd28f051ffd7676c3f · OpenDAS / vllm_cscc

[Frontend] [Core] Support for sharded tensorized models (#4990) · 51602eef

Travis Johnson authored Jun 12, 2024


Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

51602eef

tensorize_vllm_model.py 8.87 KB

Replace tensorize_vllm_model.py