---
id: pr-tensorrt-llm-12932
repo: NVIDIA/TensorRT-LLM
pr: 12932
title: '[None][feat] Add Gemma4 multimodal model support (text + vision + audio)'
author: lfr-0531
date: '2026-05-09'
url: https://github.com/NVIDIA/TensorRT-LLM/pull/12932
source_category: upstream-code
architectures:
- sm100
tags:
- attention kernel
- benchmark kernel
- cuda kernel
- moe kernel
- triton kernel
techniques: []
hardware_features: []
kernel_types:
- attention
- gemm
- moe
languages:
- cuda-cpp
- triton
captured_at: '2026-05-16'
status: merged
merge_sha: c163c728676e01693a0c0d27fd7c1127262bc15b
inclusion_reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.
changed_paths:
- tensorrt_llm/_torch/attention_backend/flashinfer.py
- tensorrt_llm/_torch/attention_backend/triton_prefill.py
- tensorrt_llm/_torch/models/__init__.py
- tensorrt_llm/_torch/models/checkpoints/__init__.py
- tensorrt_llm/_torch/models/checkpoints/hf/gemma4_weight_mapper.py
- tensorrt_llm/_torch/models/modeling_clip.py
- tensorrt_llm/_torch/models/modeling_gemma4.py
- tensorrt_llm/_torch/models/modeling_gemma4mm.py
- tensorrt_llm/_torch/models/modeling_llama.py
- tensorrt_llm/_torch/models/modeling_utils.py
- tensorrt_llm/_torch/modules/fused_moe/moe_op_backend.py
- tensorrt_llm/_torch/pyexecutor/_util.py
- tensorrt_llm/_torch/pyexecutor/config_utils.py
- tensorrt_llm/_torch/pyexecutor/model_engine.py
- tensorrt_llm/_torch/pyexecutor/resource_manager.py
- tensorrt_llm/_torch/visual_gen/models/wan/transformer_wan.py
- tensorrt_llm/commands/eval.py
- tensorrt_llm/evaluate/__init__.py
- tensorrt_llm/evaluate/covost2.py
- tensorrt_llm/evaluate/lm_eval.py
- tensorrt_llm/evaluate/lm_eval_tasks/aime/aime26.yaml
- tensorrt_llm/evaluate/lm_eval_tasks/aime/utils.py
- tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/_template_yaml
- tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/mmmu_pro.yaml
- tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/mmmu_pro_standard_10.yaml
- tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/mmmu_pro_standard_4.yaml
- tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/utils.py
- tensorrt_llm/evaluate/mmlu.py
- tensorrt_llm/models/gpt/convert.py
- tensorrt_llm/serve/tool_parser/gemma4_parser.py
- tensorrt_llm/serve/tool_parser/tool_parser_factory.py
- tests/integration/test_lists/test-db/l0_b200.yml
- tests/unittest/_torch/attention/test_flashinfer_attention.py
- tests/unittest/_torch/attention_backend/test_triton_prefill.py
- tests/unittest/_torch/executor/test_kv_cache_estimation.py
- tests/unittest/_torch/executor/test_per_layer_head_dim.py
- tests/unittest/_torch/modeling/test_gemma4_e2e_dummy.py
- tests/unittest/_torch/modeling/test_gemma4_multimodal.py
- tests/unittest/_torch/modeling/test_modeling_gemma4.py
- tests/unittest/_torch/speculative/test_eagle3.py
- tests/unittest/llmapi/apps/test_tool_parsers.py
- tests/unittest/others/test_lm_eval.py
artifact_dir: evidence/pull-bundles/tensorrt-llm/gh-12932
head_ref_oid: 18194559472c114e6ddd6c5a055d837075639b01
---
# [None][feat] Add Gemma4 multimodal model support (text + vision + audio)

- Repository: `NVIDIA/TensorRT-LLM`
- PR: [12932](https://github.com/NVIDIA/TensorRT-LLM/pull/12932)
- Decision: `Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.`
- Reason: Search-matched CUDA kernel optimization PR; materialize diff/source before synthesis.

## Changed Paths

- `tensorrt_llm/_torch/attention_backend/flashinfer.py`
- `tensorrt_llm/_torch/attention_backend/triton_prefill.py`
- `tensorrt_llm/_torch/models/__init__.py`
- `tensorrt_llm/_torch/models/checkpoints/__init__.py`
- `tensorrt_llm/_torch/models/checkpoints/hf/gemma4_weight_mapper.py`
- `tensorrt_llm/_torch/models/modeling_clip.py`
- `tensorrt_llm/_torch/models/modeling_gemma4.py`
- `tensorrt_llm/_torch/models/modeling_gemma4mm.py`
- `tensorrt_llm/_torch/models/modeling_llama.py`
- `tensorrt_llm/_torch/models/modeling_utils.py`
- `tensorrt_llm/_torch/modules/fused_moe/moe_op_backend.py`
- `tensorrt_llm/_torch/pyexecutor/_util.py`
- `tensorrt_llm/_torch/pyexecutor/config_utils.py`
- `tensorrt_llm/_torch/pyexecutor/model_engine.py`
- `tensorrt_llm/_torch/pyexecutor/resource_manager.py`
- `tensorrt_llm/_torch/visual_gen/models/wan/transformer_wan.py`
- `tensorrt_llm/commands/eval.py`
- `tensorrt_llm/evaluate/__init__.py`
- `tensorrt_llm/evaluate/covost2.py`
- `tensorrt_llm/evaluate/lm_eval.py`
- `tensorrt_llm/evaluate/lm_eval_tasks/aime/aime26.yaml`
- `tensorrt_llm/evaluate/lm_eval_tasks/aime/utils.py`
- `tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/_template_yaml`
- `tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/mmmu_pro.yaml`
- `tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/mmmu_pro_standard_10.yaml`
- `tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/mmmu_pro_standard_4.yaml`
- `tensorrt_llm/evaluate/lm_eval_tasks/mmmu_pro/utils.py`
- `tensorrt_llm/evaluate/mmlu.py`
- `tensorrt_llm/models/gpt/convert.py`
- `tensorrt_llm/serve/tool_parser/gemma4_parser.py`
- `tensorrt_llm/serve/tool_parser/tool_parser_factory.py`
- `tests/integration/test_lists/test-db/l0_b200.yml`
- `tests/unittest/_torch/attention/test_flashinfer_attention.py`
- `tests/unittest/_torch/attention_backend/test_triton_prefill.py`
- `tests/unittest/_torch/executor/test_kv_cache_estimation.py`
- `tests/unittest/_torch/executor/test_per_layer_head_dim.py`
- `tests/unittest/_torch/modeling/test_gemma4_e2e_dummy.py`
- `tests/unittest/_torch/modeling/test_gemma4_multimodal.py`
- `tests/unittest/_torch/modeling/test_modeling_gemma4.py`
- `tests/unittest/_torch/speculative/test_eagle3.py`
- `tests/unittest/llmapi/apps/test_tool_parsers.py`
- `tests/unittest/others/test_lm_eval.py`

## Evidence Bundle

`evidence/pull-bundles/tensorrt-llm/gh-12932`
