---
id: pr-sglang-20501
repo: sgl-project/sglang
pr: 20501
title: '[Kernel] Fuse temperature + softmax in sampling for decode speedup'
author: Godmook
date: '2026-04-02'
url: https://github.com/sgl-project/sglang/pull/20501
source_category: upstream-code
architectures:
- sm100
tags:
- decode
- kernel-fusion
- tma
techniques:
- kernel-fusion
hardware_features:
- tma
kernel_types:
- decode
languages:
- python
captured_at: '2026-04-17'
status: merged
merge_sha: 7a59e05dd1eb0c9d63e1cbef101dcb497e2d36e4
inclusion_reason: kernel file changes
changed_paths:
- benchmark/kernels/bench_fused_temperature_softmax.py
- python/sglang/srt/layers/fused_sampling.py
- python/sglang/srt/layers/sampler.py
- python/sglang/srt/model_executor/model_runner.py
- test/registered/sampling/test_fused_temperature_softmax.py
head_ref_oid: e4e89546a8c2e5f3cb8f039b87b19ff0d81bc6e9
artifact_dir: evidence/pull-bundles/sglang/gh-20501
---
# [Kernel] Fuse temperature + softmax in sampling for decode speedup

- Repository: `sgl-project/sglang`
- PR: [20501](https://github.com/sgl-project/sglang/pull/20501)
- Decision: `kernel file changes`
- Reason: kernel file changes

## Changed Paths

- `benchmark/kernels/bench_fused_temperature_softmax.py`
- `python/sglang/srt/layers/fused_sampling.py`
- `python/sglang/srt/layers/sampler.py`
- `python/sglang/srt/model_executor/model_runner.py`
- `test/registered/sampling/test_fused_temperature_softmax.py`

## Evidence Bundle

`evidence/pull-bundles/sglang/gh-20501`
