---
id: pr-tensorrt-llm-11993
repo: NVIDIA/TensorRT-LLM
pr: 11993
title: '[#11694][feat] AutoDeploy: Improve the piecewise CG memory usage'
author: nvchenghaoz
date: '2026-03-18'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/11993
source_category: upstream-code
architectures:
- sm100
tags:
- MLA
- attention_kv
- benchmark_test
- gemm_quant
- moe_routing
- scheduler_autotune
techniques:
- autotuning
- fine-grained-quantization
- kernel-fusion
- tile-scheduling
hardware_features: []
kernel_types:
- attention
- gemm
- mla
- moe
- scheduler
languages:
- cuda-cpp
- triton
captured_at: '2026-05-16'
status: merged
merge_sha: 3e0ae7f95b7f6cb5a44485cd0c5681043b3a5051
inclusion_reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.
changed_paths:
- examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml
- examples/auto_deploy/model_registry/configs/qwen3.5_moe_400b.yaml
- tensorrt_llm/_torch/auto_deploy/compile/backends/torch_cudagraph.py
- tensorrt_llm/_torch/auto_deploy/compile/piecewise_runner.py
- tensorrt_llm/_torch/auto_deploy/compile/piecewise_utils.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/flashinfer_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/torch_backend_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/triton_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/trtllm_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/fla/fla_backend_delta.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/fla/fla_backend_gated_delta.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/fla/torch_backend_gated_delta.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/flashinfer_backend_mamba.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/torch_backend_mamba.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/triton_backend_mamba.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/mla/flashinfer_mla.py
- tensorrt_llm/_torch/auto_deploy/transform/library/fusion.py
- tests/unittest/auto_deploy/singlegpu/compile/test_captured_graph.py
- tests/unittest/auto_deploy/singlegpu/compile/test_piecewise_runner.py
- tests/unittest/auto_deploy/singlegpu/compile/test_piecewise_utils.py
- tests/unittest/auto_deploy/singlegpu/transformations/library/test_gemm_fusion.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/mla/torch_backend_mla.py
- tensorrt_llm/_torch/auto_deploy/transform/library/compile_model.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-11993
head_ref_oid: 5e9b731f0832a320a43357cf301d6c0dbff93573
---
# [#11694][feat] AutoDeploy: Improve the piecewise CG memory usage

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [11993](https://github.com/NVIDIA/TensorRT-LLM/pull/11993)
- Decision: `Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.`
- Reason: Preserve page/KV/layout and plan-run contracts; profile memory traffic separately from math utilization.

## Changed Paths

- `examples/auto_deploy/model_registry/configs/qwen3.5_moe_35b.yaml`
- `examples/auto_deploy/model_registry/configs/qwen3.5_moe_400b.yaml`
- `tensorrt_llm/_torch/auto_deploy/compile/backends/torch_cudagraph.py`
- `tensorrt_llm/_torch/auto_deploy/compile/piecewise_runner.py`
- `tensorrt_llm/_torch/auto_deploy/compile/piecewise_utils.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/flashinfer_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/torch_backend_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/triton_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/trtllm_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/fla/fla_backend_delta.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/fla/fla_backend_gated_delta.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/fla/torch_backend_gated_delta.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/flashinfer_backend_mamba.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/torch_backend_mamba.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/triton_backend_mamba.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/mla/flashinfer_mla.py`
- `tensorrt_llm/_torch/auto_deploy/transform/library/fusion.py`
- `tests/unittest/auto_deploy/singlegpu/compile/test_captured_graph.py`
- `tests/unittest/auto_deploy/singlegpu/compile/test_piecewise_runner.py`
- `tests/unittest/auto_deploy/singlegpu/compile/test_piecewise_utils.py`
- `tests/unittest/auto_deploy/singlegpu/transformations/library/test_gemm_fusion.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/mla/torch_backend_mla.py`
- `tensorrt_llm/_torch/auto_deploy/transform/library/compile_model.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-11993`
