v1.0

README.md

+# Gemma-3
+Gemma-3仅以27B就击败DeepSeek 671B模型，成为仅次于DeepSeek R1最优开源模型。
+
+## 论文
+`无`
+
+## 模型结构
+Gemma-3模型沿用了与前代版本相同的解码器Transformer结构，改进点为：增加了局部和全局注意力层的交错模式，减少了KV缓存的使用。此外，模型还支持128K令牌的长上下文，并在全局自注意力层上增加了RoPE基频率，从1万次提高到1百万次，并保持局部层的频率为1万次。
+<div align=center>
+    <img src="./doc/gemma2.png"/>
+</div>
+
+## 算法原理
+基于人类反馈的强化学习（RLHF）使模型预测与人类偏好保持一致，机器反馈强化学习（RLMF）增强数学推理，强化学习执行反馈（RLEF），提高编码能力。
+<div align=center>
+    <img src="./doc/RLHF.png"/>
+</div>
+
+## 环境配置
+```
+mv Gemma-3_pytorch Gemma-3 # 去框架名后缀
+```
+
+### Docker（方法一）
+```
+docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.4.1-ubuntu22.04-dtk25.04-py3.10-fixpy
+# <your IMAGE ID>为以上拉取的docker的镜像ID替换，本镜像为：e77c15729879
+docker run -it --shm-size=64G --network host -v $PWD/Gemma-3:/home/Gemma-3 -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name gemma3 <your IMAGE ID> bash
+cd /home/Gemma-3
+unzip 46350f5eae87ac1d168ddfdc57a0b39b64b9a029.zip
+cd transformers-46350f5eae87ac1d168ddfdc57a0b39b64b9a029
+pip install -e . # 本项目作者只支持transformers==4.50.0.dev0
+```
+### Dockerfile（方法二）
+```
+cd /home/Gemma-3/docker
+docker build --no-cache -t gemma3:latest .
+docker run --shm-size=64G --network host --name gemma3 -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video -v $PWD/../../Gemma-3:/home/Gemma-3 -it gemma3 bash
+# 若遇到Dockerfile启动的方式安装环境需要长时间等待，可注释掉里面的pip安装，启动容器后再安装python库：pip install -r requirements.txt。
+cd /home/Gemma-3
+unzip 46350f5eae87ac1d168ddfdc57a0b39b64b9a029.zip
+cd transformers-46350f5eae87ac1d168ddfdc57a0b39b64b9a029
+pip install -e . # 本项目作者只支持transformers==4.50.0.dev0
+```
+### Anaconda（方法三）
+1、关于本项目DCU显卡所需的特殊深度学习库可从光合开发者社区下载安装：
+- https://developer.hpccube.com/tool/
+```
+DTK驱动:dtk2504
+python:python3.10
+torch:2.4.1
+torchvision:0.19.1
+triton:3.0.0
+vllm:0.6.2
+flash-attn:2.6.1
+deepspeed:0.14.2
+apex:1.4.0
+transformers:4.50.0.dev0
+```
+
+`Tips：以上dtk驱动、python、torch等DCU相关工具版本需要严格一一对应。`
+
+2、其它非特殊库参照requirements.txt安装
+```
+cd /home/Gemma-3
+unzip 46350f5eae87ac1d168ddfdc57a0b39b64b9a029.zip
+cd transformers-46350f5eae87ac1d168ddfdc57a0b39b64b9a029
+pip install -e . # 本项目作者只支持transformers==4.50.0.dev0
+```
+
+## 数据集
+`无`
+
+## 训练
+无
+
+## 推理
+预训练权重目录结构：
+```
+/home/Gemma-3
+    └── google/gemma-3-27b-it
+``` 
+
+### 单机多卡
+```
+# transformers推理
+python infer_transformers.py 
+
+# VLLM推理的适配涉及底层需求较多，敬请期待陆续开放。
+```
+
+## result
+
+`输入: `
+```
+"content": [{"type": "image", "image": "./candy.JPG"},
+            {"type": "text", "text": "Describe this image in detail."}]
+```
+
+`输出:`
+```
+Here's a detailed description of the image:
+
+**Overall Impression:**
+
+The image shows a person's open palm holding four colorful, round objects that appear to be decorative stones or beads. The background is slightly blurred, suggesting a focus on the hand and the objects within it.
+
+**Detailed Description:**
+
+*   **Hand:** The hand is fair-skinned and appears to be an adult's. It's open, with the palm facing upwards, displaying the four objects.
+```
+
+### 精度
+DCU与GPU精度一致，推理框架：pytorch。
+
+## 应用场景
+### 算法类别
+`对话问答`
+### 热点应用行业
+`制造,广媒,金融,能源,医疗,家居,教育`
+## 预训练权重
+预训练权重快速下载中心：[SCNet AIModels](http://113.200.138.88:18080/aimodels) ，项目中的预训练权重可从快速下载通道下载：[gemma-3-27b-it](http://113.200.138.88:18080/aimodels/google/gemma-3-27b-it.git)
+
+HF社区下载地址为：[gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it)
+## 源码仓库及问题反馈
+- http://developer.sourcefind.cn/codes/modelzoo/Gemma-3_pytorch.git
+## 参考资料
+- https://huggingface.co/google/gemma-3-27b-it
+

docker/Dockerfile

+FROM image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.4.1-ubuntu22.04-dtk25.04-py3.10-fixpy
+ENV DEBIAN_FRONTEND=noninteractive
+# RUN yum update && yum install -y git cmake wget build-essential
+# RUN source /opt/dtk-dtk25.04/env.sh
+# # 安装pip相关依赖
+COPY requirements.txt requirements.txt
+RUN pip3 install -r requirements.txt -i http://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com
+

docker/requirements.txt

+

infer_transformers.py

+# pip install accelerate
+
+from transformers import AutoProcessor, Gemma3ForConditionalGeneration
+from PIL import Image
+import requests
+import torch
+
+model_id = "google/gemma-3-27b-it"
+
+model = Gemma3ForConditionalGeneration.from_pretrained(
+    model_id, device_map="auto"
+).eval()
+
+processor = AutoProcessor.from_pretrained(model_id)
+
+messages = [
+    {
+        "role": "system",
+        "content": [{"type": "text", "text": "You are a helpful assistant."}]
+    },
+    {
+        "role": "user",
+        "content": [
+            {"type": "image", "image": "./candy.JPG"},
+            {"type": "text", "text": "Describe this image in detail."}
+        ]
+    }
+]
+
+inputs = processor.apply_chat_template(
+    messages, add_generation_prompt=True, tokenize=True,
+    return_dict=True, return_tensors="pt"
+).to(model.device, dtype=torch.bfloat16)
+
+input_len = inputs["input_ids"].shape[-1]
+
+with torch.inference_mode():
+    generation = model.generate(**inputs, max_new_tokens=100, do_sample=False)
+    generation = generation[0][input_len:]
+
+decoded = processor.decode(generation, skip_special_tokens=True)
+print(decoded)
+
+# **Overall Impression:** The image is a close-up shot of a vibrant garden scene, 
+# focusing on a cluster of pink cosmos flowers and a busy bumblebee. 
+# It has a slightly soft, natural feel, likely captured in daylight.
+

model.properties

+# 模型编码
+modelCode=1469
+# 模型名称
+modelName=Gemma-3_pytorch
+# 模型描述
+modelDescription=Gemma-3仅以27B就击败DeepSeek 671B模型，成为仅次于DeepSeek R1最优开源模型。
+# 应用场景
+appScenario=推理,对话问答,制造,广媒,金融,能源,医疗,家居,教育
+# 框架类型
+frameType=pytorch