vllm/worker/cpu_pooling_model_runner.py · f256ebe4df6757d76f1f1642d7e110268a2f8190 · OpenDAS / vllm_cscc · GitLab

Find file Blame History Permalink

[torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
cf5f000d

cpu_pooling_model_runner.py 5.2 KB