GitLab

新增环境变量 VLLM_V1_USE_REDUCED_TOPK_TOPP_SAMPLER 用于开关控制
扩展 SamplingMetadata，增加 max_top_k 与 has_any_no_top_k
在 InputBatch 侧计算 top-k 的主机端汇总信息，避免 device 同步
更新 Sampler/TopKTopPSampler 传递并使用新参数以启用优化采样

- 在 InputBatch.refresh_metadata 中为展开后的采样元数据引入 repeat_count 记录重复次数
- 完善元数据刷新逻辑以适配 reject-sampling 优化路径
- 更新 GPUModelRunnerBase，在 batch 处理阶段正确消费新的采样元数据与重复计数

Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Andrew Feldman <afeldman@redhat.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Isotr0py <2037008807@qq.com>

Signed-off-by: Max de Bayser <mbayser@br.ibm.com>
Signed-off-by: Max de Bayser <maxdebayser@gmail.com>
Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: 22quinn <33176974+22quinn@users.noreply.github.com>

Signed-off-by: Andrew Feldman <afeldman@redhat.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: simon-mo <simon.mo@hey.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Mark McLoughlin <markmc@redhat.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: 22quinn <33176974+22quinn@users.noreply.github.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Lucas Wilkinson <lwilkins@redhat.com>

Signed-off-by: Nick Hill <nhill@redhat.com>
Co-authored-by: Ye Cao <caoye.cao@alibaba-inc.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>

Signed-off-by: Lu Fang <lufang@fb.com>

Signed-off-by: Chen Zhang <zhangch99@outlook.com>

Signed-off-by: Yang Chen <yangche@fb.com>

Signed-off-by: Lu Fang <lufang@fb.com>

Signed-off-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>

Signed-off-by: LiuXiaoxuanPKU <lilyliupku@gmail.com>

Signed-off-by: Aoyu <aoyuzhan@amazon.com>
Co-authored-by: Aoyu <aoyuzhan@amazon.com>

This PR is adding support for sample logprobs & prompt logprobs to vLLM v1.

New behavior:

- During model execution, model runner computes sample logprobs (if user-provided logprobs setting is not None) and prompt logprobs (if user-provided prompt_logprobs setting is not None). For both sample and prompt logprobs, the engine core returns 3 vectors: token ids, token logprob values, token ranks. Ranks reflect tokens' 1-indexed positions in the vocabulary vector after sorting the vocabulary by log probability in descending order.
- In scheduler.update_from_output(), sample and prompt logprobs are incorporated into the EngineCoreOutput data structure which is transferred to the engine client. If multiprocessing is enabled, then sample and prompt logprobs will be (de)serialized when the EngineCoreOutput data structure is (de)serialized.
- During output processing, the LogprobsProcessor transforms the triplet of token ids, token logprobs values, and token ranks into the OpenAI-compatible List[Dict[token id,Logprob]] format (for sample and prompt logprobs respectively.)
- Each Logprob instance (whether sample- or prompt-) consists of a token's log-probability, rank, and detokenized string representation. Note that logprob detokenization is handled by the LogprobsProcessor not the detokenizer.
Signed-off-by: Andrew Feldman <afeldman@neuralmagic.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Nick Hill <nhill@redhat.com>

Signed-off-by: Varun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: Varun Sundar Rabindranath <varun@neuralmagic.com>