vllm/worker/model_runner.py · 51602eefd38250325e541abd28f051ffd7676c3f · OpenDAS / vllm_cscc

[Frontend] [Core] Support for sharded tensorized models (#4990) · 51602eef

Travis Johnson authored Jun 12, 2024


Signed-off-by: Travis Johnson <tsjohnso@us.ibm.com>
Co-authored-by: Sanger Steel <sangersteel@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>

51602eef

model_runner.py 47 KB

Replace model_runner.py