"src/targets/cpu/cpu_lowering.cpp" did not exist on "f199ea9ef61c928ea8b35f6f22cd902e2b619840"
Defensively copy `sampling_params` (#2881)
If the SamplingParams object passed to LLMEngine.add_request() is mutated after it returns, it could affect the async sampling process for that request. Suggested by @Yard1 https://github.com/vllm-project/vllm/pull/2514#discussion_r1490106059
Showing
Please register or sign in to comment