---
id: pr-tensorrt-llm-12861
repo: NVIDIA/TensorRT-LLM
pr: 12861
title: '[None][feat] AutoDeploy: Add Gemma4 vision support'
author: bmarimuthu-nv
date: '2026-05-04'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/12861
source_category: upstream-code
architectures:
- sm100
tags:
- triton kernel
techniques: []
hardware_features: []
kernel_types:
- gemm
languages:
- cuda-cpp
- triton
captured_at: '2026-05-16'
status: merged
merge_sha: fd6996106347e58e657e36f4ed7f5bc0348f1d1f
inclusion_reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.
changed_paths:
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/torch_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/torch_backend_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention/triton_paged_attention.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/attention_interface.py
- tensorrt_llm/_torch/auto_deploy/custom_ops/semantic_mask_registry.py
- tensorrt_llm/_torch/auto_deploy/llm.py
- tensorrt_llm/_torch/auto_deploy/models/custom/modeling_gemma4.py
- tensorrt_llm/_torch/auto_deploy/shim/ad_executor.py
- tensorrt_llm/_torch/auto_deploy/transform/__init__.py
- tensorrt_llm/_torch/auto_deploy/transform/library/kvcache.py
- tensorrt_llm/_torch/auto_deploy/transform/library/kvcache_transformers.py
- tests/integration/defs/accuracy/test_llm_api_autodeploy.py
- tests/unittest/_torch/auto_deploy/unit/singlegpu/models/test_gemma4_modeling.py
- tests/unittest/auto_deploy/_utils_test/torch_attention_reference.py
- tests/unittest/auto_deploy/singlegpu/custom_ops/attention/test_torch_attention_op.py
- tests/unittest/auto_deploy/singlegpu/custom_ops/attention/test_triton_paged_attention.py
- tests/unittest/auto_deploy/singlegpu/transformations/library/test_kv_cache.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-12861
head_ref_oid: 6d5e898a5383e400a8f75afce258ce98a39637a2
---
# [None][feat] AutoDeploy: Add Gemma4 vision support

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [12861](https://github.com/NVIDIA/TensorRT-LLM/pull/12861)
- Decision: `Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.`
- Reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.

## Changed Paths

- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/torch_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/torch_backend_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention/triton_paged_attention.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/attention_interface.py`
- `tensorrt_llm/_torch/auto_deploy/custom_ops/semantic_mask_registry.py`
- `tensorrt_llm/_torch/auto_deploy/llm.py`
- `tensorrt_llm/_torch/auto_deploy/models/custom/modeling_gemma4.py`
- `tensorrt_llm/_torch/auto_deploy/shim/ad_executor.py`
- `tensorrt_llm/_torch/auto_deploy/transform/__init__.py`
- `tensorrt_llm/_torch/auto_deploy/transform/library/kvcache.py`
- `tensorrt_llm/_torch/auto_deploy/transform/library/kvcache_transformers.py`
- `tests/integration/defs/accuracy/test_llm_api_autodeploy.py`
- `tests/unittest/_torch/auto_deploy/unit/singlegpu/models/test_gemma4_modeling.py`
- `tests/unittest/auto_deploy/_utils_test/torch_attention_reference.py`
- `tests/unittest/auto_deploy/singlegpu/custom_ops/attention/test_torch_attention_op.py`
- `tests/unittest/auto_deploy/singlegpu/custom_ops/attention/test_triton_paged_attention.py`
- `tests/unittest/auto_deploy/singlegpu/transformations/library/test_kv_cache.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-12861`
