vllm/worker/pooling_model_runner.py · afd0da2186c1d58fb48e138df0a2f548612b5d7d · OpenDAS / vllm_cscc · GitLab

Find file Blame History Permalink

[torch.compile] Hide KV cache behind torch.compile boundary (#11677) · cf5f000d
Chen Zhang authored Jan 10, 2025
```
Signed-off-by: Chen Zhang <zhangch99@outlook.com>
```
cf5f000d

pooling_model_runner.py 8.5 KB