---
id: pr-tensorrt-llm-12062
repo: NVIDIA/TensorRT-LLM
pr: 12062
title: '[TRTLLM-11540][feat] Add EAGLE3 dynamic tree speculative decoding support'
author: sunnyqgg
date: '2026-04-12'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/12062
source_category: upstream-code
architectures:
- sm100
tags:
- attention_kv
- cuda kernel
- memory_primitives
techniques:
- cache-policy
hardware_features: []
kernel_types:
- attention
- decode
- memory
languages:
- cuda-cpp
captured_at: '2026-05-16'
status: merged
merge_sha: 4ece13c0bf35553e5a339e1f2ce8ebc5cf9e99d8
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- cpp/tensorrt_llm/kernels/speculativeDecoding/dynamicTreeKernels.cu
- cpp/tensorrt_llm/kernels/speculativeDecoding/dynamicTreeKernels.h
- cpp/tensorrt_llm/kernels/speculativeDecoding/kvCacheUpdateKernels.cu
- cpp/tensorrt_llm/kernels/speculativeDecoding/kvCacheUpdateKernels.h
- cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaRunnerParams.h
- cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/prepareCustomMask.cu
- cpp/tensorrt_llm/kernels/xqaDispatcher.cpp
- cpp/tensorrt_llm/thop/CMakeLists.txt
- cpp/tensorrt_llm/thop/dynamicTreeOp.cpp
- cpp/tensorrt_llm/thop/parallelDecodeKVCacheUpdateOp.cpp
- docs/source/features/feature-combination-matrix.md
- docs/source/features/speculative-decoding.md
- docs/source/models/supported-models.md
- examples/llm-api/quickstart_advanced.py
- tensorrt_llm/_torch/attention_backend/interface.py
- tensorrt_llm/_torch/attention_backend/sparse/dsa.py
- tensorrt_llm/_torch/attention_backend/trtllm.py
- tensorrt_llm/_torch/modules/attention.py
- tests/unittest/_torch/modeling/test_modeling_llama.py
- tests/unittest/_torch/speculative/test_draft_token_prepare_for_generation.py
- tensorrt_llm/_torch/pyexecutor/model_engine.py
- tensorrt_llm/_torch/pyexecutor/py_executor.py
- tensorrt_llm/_torch/pyexecutor/py_executor_creator.py
- tensorrt_llm/_torch/pyexecutor/resource_manager.py
- tensorrt_llm/_torch/speculative/drafting_loops.py
- tensorrt_llm/_torch/speculative/dynamic_tree_ops.py
- tensorrt_llm/_torch/speculative/eagle3.py
- tensorrt_llm/_torch/speculative/eagle3_dynamic_tree.py
- tensorrt_llm/_torch/speculative/spec_sampler_base.py
- tensorrt_llm/_torch/speculative/spec_tree_manager.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-12062
head_ref_oid: 9a9cc5edd7dd8946db533f4c032ba69009db544c
---
# [TRTLLM-11540][feat] Add EAGLE3 dynamic tree speculative decoding support

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [12062](https://github.com/NVIDIA/TensorRT-LLM/pull/12062)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `cpp/tensorrt_llm/kernels/speculativeDecoding/dynamicTreeKernels.cu`
- `cpp/tensorrt_llm/kernels/speculativeDecoding/dynamicTreeKernels.h`
- `cpp/tensorrt_llm/kernels/speculativeDecoding/kvCacheUpdateKernels.cu`
- `cpp/tensorrt_llm/kernels/speculativeDecoding/kvCacheUpdateKernels.h`
- `cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/fmhaRunnerParams.h`
- `cpp/tensorrt_llm/kernels/trtllmGenKernels/fmha/prepareCustomMask.cu`
- `cpp/tensorrt_llm/kernels/xqaDispatcher.cpp`
- `cpp/tensorrt_llm/thop/CMakeLists.txt`
- `cpp/tensorrt_llm/thop/dynamicTreeOp.cpp`
- `cpp/tensorrt_llm/thop/parallelDecodeKVCacheUpdateOp.cpp`
- `docs/source/features/feature-combination-matrix.md`
- `docs/source/features/speculative-decoding.md`
- `docs/source/models/supported-models.md`
- `examples/llm-api/quickstart_advanced.py`
- `tensorrt_llm/_torch/attention_backend/interface.py`
- `tensorrt_llm/_torch/attention_backend/sparse/dsa.py`
- `tensorrt_llm/_torch/attention_backend/trtllm.py`
- `tensorrt_llm/_torch/modules/attention.py`
- `tests/unittest/_torch/modeling/test_modeling_llama.py`
- `tests/unittest/_torch/speculative/test_draft_token_prepare_for_generation.py`
- `tensorrt_llm/_torch/pyexecutor/model_engine.py`
- `tensorrt_llm/_torch/pyexecutor/py_executor.py`
- `tensorrt_llm/_torch/pyexecutor/py_executor_creator.py`
- `tensorrt_llm/_torch/pyexecutor/resource_manager.py`
- `tensorrt_llm/_torch/speculative/drafting_loops.py`
- `tensorrt_llm/_torch/speculative/dynamic_tree_ops.py`
- `tensorrt_llm/_torch/speculative/eagle3.py`
- `tensorrt_llm/_torch/speculative/eagle3_dynamic_tree.py`
- `tensorrt_llm/_torch/speculative/spec_sampler_base.py`
- `tensorrt_llm/_torch/speculative/spec_tree_manager.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-12062`
