---
id: pr-tensorrt-llm-13658
repo: NVIDIA/TensorRT-LLM
pr: 13658
title: '[None][test] add Nemotron Ultra V3 AutoDeploy accuracy test'
author: tcherckez-nvidia
date: '2026-05-12'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/13658
source_category: upstream-code
architectures:
- sm100
tags:
- arch_pipeline
- attention_kv
- benchmark_test
- blackwell
- nvfp4
- scheduler_autotune
techniques:
- autotuning
- pipeline-stages
- tile-scheduling
hardware_features:
- blackwell
- fp4
- nvfp4
kernel_types:
- attention
- runtime
- scheduler
languages: []
captured_at: '2026-05-16'
status: merged
merge_sha: 113bc640bc37932328a83a2a72d45e7103290512
inclusion_reason: Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.
changed_paths:
- examples/auto_deploy/model_registry/configs/ultra_v3.yaml
- tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/flashinfer_backend_mamba.py
- tests/integration/defs/accuracy/references/gsm8k.yaml
- tests/integration/defs/accuracy/references/mmlu.yaml
- tests/integration/defs/accuracy/test_llm_api_autodeploy.py
- tests/integration/test_lists/test-db/l0_dgx_b200.yml
- tests/test_common/llm_data.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-13658
head_ref_oid: 312bdb83bea4a9b52682653372678ba97418252e
---
# [None][test] add Nemotron Ultra V3 AutoDeploy accuracy test

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [13658](https://github.com/NVIDIA/TensorRT-LLM/pull/13658)
- Decision: `Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.`
- Reason: Mine shape sets, tolerance rules, warmup logic, and profile commands before mutating code.

## Changed Paths

- `examples/auto_deploy/model_registry/configs/ultra_v3.yaml`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/mamba/flashinfer_backend_mamba.py`
- `tests/integration/defs/accuracy/references/gsm8k.yaml`
- `tests/integration/defs/accuracy/references/mmlu.yaml`
- `tests/integration/defs/accuracy/test_llm_api_autodeploy.py`
- `tests/integration/test_lists/test-db/l0_dgx_b200.yml`
- `tests/test_common/llm_data.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-13658`
