Skip to content

GitLab

Commit 79606193 authored by chenpangpang's avatar chenpangpang
Browse files

feat: 增加dcu-tesorflow镜像检测脚本

parent e0613b00
Hide whitespace changes
Inline Side-by-side
Showing with 681 additions and 0 deletions
dcu/tensorflow_check.ipynb 0 → 100644
View file @ 79606193
{
"cells": [
{
"cell_type": "markdown",
"id": "1480baa1-193b-4d32-8458-480cb2864ba0",
"metadata": {},
"source": [
"# DCU镜像检测-tensorflow\n",
"此脚本为tensorflow的DCU镜像通用检查脚本，更多项目和issue信息请参考[链接](https://developer.sourcefind.cn/codes/chenpangpang/gpu-base-image-test)。\n",
"## 一、智能计算-线上检测\n",
"此部分是根据[智能计算文档](https://r0ddbu55vzx.feishu.cn/wiki/RvepwMCayi9dY4ktMquc3oiPnVl)中的*线上检测*来进行镜像检查，预期结果:\n",
"1. `layer_check_tf.__dcu_version__`：dtk版本与镜像名称中的dtk版本一致；\n",
"2. `tf.test.is_gpu_available()`：结果为`True`。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "55fd2be3-3d58-45b2-ae4d-01d02d0cfb20",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['/opt/conda/lib/python38.zip', '/opt/conda/lib/python3.8', '/opt/conda/lib/python3.8/lib-dynload', '', '/opt/conda/lib/python3.8/site-packages']\n",
"1.2.0 \n",
" 1.2.0+das.dtk24041\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-12 14:21:00.451531: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
"To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"WARNING:tensorflow:From /tmp/ipykernel_5101/616564938.py:6: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use `tf.config.list_physical_devices('GPU')` instead.\n",
"True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-12 14:21:02.635306: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Created device /device:GPU:0 with 63225 MB memory: -> device: 0, name: DCU K100_AI, pci bus id: 0000:16:00.0\n"
]
}
],
"source": [
"import sys\n",
"print(sys.path)\n",
"import layer_check_tf\n",
"print(layer_check_tf.__version__,'\\n',layer_check_tf.__dcu_version__)\n",
"import tensorflow as tf\n",
"print(tf.test.is_gpu_available())"
]
},
{
"cell_type": "markdown",
"id": "06b640a9-ac50-4c73-94fe-07a2789a59f3",
"metadata": {},
"source": [
"## 二、曙光智算-通用镜像检测\n",
"此部分是根据曙光智能开发的通用镜像检测方法来进行镜像检查，共分为版本检查、文本模型检查、图像模型检查三个部分。\n",
"### 2.1 版本检查\n",
"预期结果：\n",
"1. 系统版本和镜像名称中的操作系统一致；\n",
"2. python版本和镜像名称中的python版本一致；\n",
"3. tensorflow和镜像名称中的tensorflow版本一致。"
]
},
{
"cell_type": "code",
"execution_count": 8,
"id": "642d011c-eb3f-4aea-9fff-cb75b2558913",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Ubuntu 22.04.5 LTS \\n \\l\n",
"\n",
"python version: 3.8.20 (default, Oct 3 2024, 15:24:27) \n",
"[GCC 11.2.0]\n",
"tensorflow version: 2.13.1\n",
"tensorflow cuda available: True\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-12 14:46:31.245907: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Created device /device:GPU:0 with 63225 MB memory: -> device: 0, name: DCU K100_AI, pci bus id: 0000:16:00.0\n",
"sh: line 1: nvcc: command not found\n"
]
},
{
"data": {
"text/plain": [
"0"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import os\n",
"os.system(\"cat /etc/issue\")\n",
"import sys\n",
"print(\"python version: \", sys.version)\n",
"import tensorflow as tf\n",
"print(\"tensorflow version: \", tf.__version__)"
]
},
{
"cell_type": "markdown",
"id": "3a987641-f8a1-4b4c-b393-a510f35e3710",
"metadata": {},
"source": [
"### 2.2 文本模型检查\n",
"预期结果：\n",
"1. 程序正常运行；\n",
"2. 输出tenosr不全为0。"
]
},
{
"cell_type": "markdown",
"id": "1d8ba587-f615-45f3-98ce-90879b6183aa",
"metadata": {},
"source": [
"步骤一 安装相关依赖库"
]
},
{
"cell_type": "code",
"execution_count": 5,
"id": "fb44e957-de9b-4b54-a736-d46fe3ce1634",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple\n",
"Requirement already satisfied: tensorflow-hub in /opt/conda/lib/python3.8/site-packages (0.16.1)\n",
"Collecting tensorflow-text\n",
" Downloading https://pypi.tuna.tsinghua.edu.cn/packages/70/f3/a2e79de3c1255313f722c071602bc36b4ac52a1199d3b82dc0d641c2f9b3/tensorflow_text-2.13.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (6.5 MB)\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m6.5/6.5 MB\u001b[0m \u001b[31m14.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0ma \u001b[36m0:00:01\u001b[0m\n",
"\u001b[?25hRequirement already satisfied: numpy>=1.12.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow-hub) (1.24.3)\n",
"Requirement already satisfied: protobuf>=3.19.6 in /opt/conda/lib/python3.8/site-packages (from tensorflow-hub) (4.25.5)\n",
"Requirement already satisfied: tf-keras>=2.14.1 in /opt/conda/lib/python3.8/site-packages (from tensorflow-hub) (2.15.0)\n",
"Requirement already satisfied: tensorflow<2.14,>=2.13.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow-text) (2.13.1+das.opt1.dtk24042)\n",
"Requirement already satisfied: absl-py>=1.0.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (2.1.0)\n",
"Requirement already satisfied: astunparse>=1.6.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (1.6.3)\n",
"Requirement already satisfied: flatbuffers>=23.1.21 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (24.3.25)\n",
"Requirement already satisfied: gast<=0.4.0,>=0.2.1 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (0.4.0)\n",
"Requirement already satisfied: google-pasta>=0.1.1 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (0.2.0)\n",
"Requirement already satisfied: h5py>=2.9.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (3.11.0)\n",
"Requirement already satisfied: libclang>=13.0.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (18.1.1)\n",
"Requirement already satisfied: opt-einsum>=2.3.2 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (3.4.0)\n",
"Requirement already satisfied: packaging in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (24.1)\n",
"Requirement already satisfied: setuptools in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (65.5.0)\n",
"Requirement already satisfied: six>=1.12.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (1.16.0)\n",
"Requirement already satisfied: termcolor>=1.1.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (2.4.0)\n",
"Collecting typing-extensions<4.6.0,>=3.6.6 (from tensorflow<2.14,>=2.13.0->tensorflow-text)\n",
" Downloading https://pypi.tuna.tsinghua.edu.cn/packages/31/25/5abcd82372d3d4a3932e1fa8c3dbf9efac10cc7c0d16e78467460571b404/typing_extensions-4.5.0-py3-none-any.whl (27 kB)\n",
"Requirement already satisfied: wrapt>=1.11.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (1.16.0)\n",
"Requirement already satisfied: grpcio<2.0,>=1.24.3 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (1.66.2)\n",
"Requirement already satisfied: tensorboard<2.14,>=2.13 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (2.13.0)\n",
"Requirement already satisfied: tensorflow-estimator<2.14,>=2.13.0 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (2.13.0)\n",
"Requirement already satisfied: keras<2.14,>=2.13.1 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (2.13.1)\n",
"Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /opt/conda/lib/python3.8/site-packages (from tensorflow<2.14,>=2.13.0->tensorflow-text) (0.34.0)\n",
"Requirement already satisfied: wheel<1.0,>=0.23.0 in /opt/conda/lib/python3.8/site-packages (from astunparse>=1.6.0->tensorflow<2.14,>=2.13.0->tensorflow-text) (0.37.1)\n",
"Requirement already satisfied: google-auth<3,>=1.6.3 in /opt/conda/lib/python3.8/site-packages (from tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (2.35.0)\n",
"Requirement already satisfied: google-auth-oauthlib<1.1,>=0.5 in /opt/conda/lib/python3.8/site-packages (from tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (1.0.0)\n",
"Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.8/site-packages (from tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (3.7)\n",
"Requirement already satisfied: requests<3,>=2.21.0 in /opt/conda/lib/python3.8/site-packages (from tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (2.32.3)\n",
"Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /opt/conda/lib/python3.8/site-packages (from tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (0.7.2)\n",
"Requirement already satisfied: werkzeug>=1.0.1 in /opt/conda/lib/python3.8/site-packages (from tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (3.0.4)\n",
"Requirement already satisfied: cachetools<6.0,>=2.0.0 in /opt/conda/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (5.5.0)\n",
"Requirement already satisfied: pyasn1-modules>=0.2.1 in /opt/conda/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (0.4.1)\n",
"Requirement already satisfied: rsa<5,>=3.1.4 in /opt/conda/lib/python3.8/site-packages (from google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (4.9)\n",
"Requirement already satisfied: requests-oauthlib>=0.7.0 in /opt/conda/lib/python3.8/site-packages (from google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (2.0.0)\n",
"Requirement already satisfied: importlib-metadata>=4.4 in /opt/conda/lib/python3.8/site-packages (from markdown>=2.6.8->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (8.5.0)\n",
"Requirement already satisfied: charset-normalizer<4,>=2 in /opt/conda/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (3.3.2)\n",
"Requirement already satisfied: idna<4,>=2.5 in /opt/conda/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (3.7)\n",
"Requirement already satisfied: urllib3<3,>=1.21.1 in /opt/conda/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (2.2.3)\n",
"Requirement already satisfied: certifi>=2017.4.17 in /opt/conda/lib/python3.8/site-packages (from requests<3,>=2.21.0->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (2024.8.30)\n",
"Requirement already satisfied: MarkupSafe>=2.1.1 in /opt/conda/lib/python3.8/site-packages (from werkzeug>=1.0.1->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (2.1.5)\n",
"Requirement already satisfied: zipp>=3.20 in /opt/conda/lib/python3.8/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (3.20.2)\n",
"Requirement already satisfied: pyasn1<0.7.0,>=0.4.6 in /opt/conda/lib/python3.8/site-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (0.6.1)\n",
"Requirement already satisfied: oauthlib>=3.0.0 in /opt/conda/lib/python3.8/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<1.1,>=0.5->tensorboard<2.14,>=2.13->tensorflow<2.14,>=2.13.0->tensorflow-text) (3.2.2)\n",
"Installing collected packages: typing-extensions, tensorflow-text\n",
" Attempting uninstall: typing-extensions\n",
" Found existing installation: typing_extensions 4.6.0\n",
" Uninstalling typing_extensions-4.6.0:\n",
" Successfully uninstalled typing_extensions-4.6.0\n",
"Successfully installed tensorflow-text-2.13.0 typing-extensions-4.5.0\n",
"\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager, possibly rendering your system unusable.It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv. Use the --root-user-action option if you know what you are doing and want to suppress this warning.\u001b[0m\u001b[33m\n",
"\u001b[0m"
]
}
],
"source": [
"!pip install tensorflow-hub tensorflow-text"
]
},
{
"cell_type": "markdown",
"id": "f6e66aa5-902d-43da-b258-7174462f6b04",
"metadata": {},
"source": [
"步骤二 运行程序"
]
},
{
"cell_type": "code",
"execution_count": 6,
"id": "e85a3b8b-7103-4308-91ac-4e9417c58db9",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-12 14:24:50.568152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 63225 MB memory: -> device: 0, name: DCU K100_AI, pci bus id: 0000:16:00.0\n",
"2024-10-12 14:24:51.532740: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:24:53.060138: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:24:53.073139: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:24:53.081631: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:24:53.083330: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:24:53.095114: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:24:53.095851: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:24:53.111755: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.792283: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.794921: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.796453: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.797443: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.799239: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.800131: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.801076: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.802493: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.803379: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.804269: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.833876: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:45.835112: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:46.879646: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:47.142667: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:47.720696: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:47.721894: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:47.722876: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:47.723797: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:47.743944: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:47.745148: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.459410: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.461444: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.462579: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.464765: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.468979: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.474551: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.475852: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.479175: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:48.549247: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:49.036286: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:49.151861: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 14:25:49.642933: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[-8.97941411e-01 -4.21122879e-01 -7.14764476e-01 7.73540258e-01\n",
" 5.23896396e-01 -2.22132996e-01 8.59079421e-01 2.56452531e-01\n",
" -6.35726333e-01 -9.99986947e-01 -3.78803968e-01 8.12413812e-01\n",
" 9.83724535e-01 2.40791693e-01 9.18967426e-01 -6.08405709e-01\n",
" -2.06733987e-01 -5.68343341e-01 3.09819341e-01 -4.92150992e-01\n",
" 6.79648817e-01 9.99840379e-01 4.06326801e-01 3.17327946e-01\n",
" 5.78657150e-01 9.68130469e-01 -7.63652146e-01 9.27225411e-01\n",
" 9.53164160e-01 6.58057094e-01 -6.57289743e-01 2.50330538e-01\n",
" -9.88110483e-01 -2.39297360e-01 -7.63934493e-01 -9.91781354e-01\n",
" 4.83872026e-01 -7.73471475e-01 7.10693537e-04 -1.34690497e-02\n",
" -8.96146715e-01 3.51920605e-01 9.99937832e-01 -9.21932980e-02\n",
" 3.70957375e-01 -3.04825932e-01 -1.00000000e+00 2.35189080e-01\n",
" -8.96254599e-01 7.68426239e-01 6.41088367e-01 6.17559850e-01\n",
" 1.59086362e-01 4.23474669e-01 4.94476110e-01 2.82408863e-01\n",
" 6.79241493e-05 1.54811859e-01 -2.84662396e-01 -6.06176138e-01\n",
" -6.23841047e-01 4.29913312e-01 -6.78956032e-01 -9.37227011e-01\n",
" 6.83781326e-01 5.16698062e-01 -1.34410024e-01 -3.55892628e-01\n",
" -1.07093759e-01 -1.91312894e-01 8.62805367e-01 2.46813089e-01\n",
" 2.34420583e-01 -8.53678703e-01 4.25712347e-01 2.89026469e-01\n",
" -6.34167969e-01 1.00000000e+00 -5.62016070e-01 -9.83899951e-01\n",
" 6.55205548e-01 5.85005105e-01 5.22590816e-01 5.35732619e-02\n",
" 1.80098549e-01 -1.00000000e+00 5.28661907e-01 -1.80297807e-01\n",
" -9.91242945e-01 1.91021845e-01 5.38162053e-01 -1.88897073e-01\n",
" 1.40862346e-01 5.71373522e-01 -5.10222971e-01 -3.00845534e-01\n",
" -3.42853099e-01 -6.14215076e-01 -3.98965597e-01 -3.43374252e-01\n",
" 1.04598977e-01 -2.26879060e-01 -3.27940404e-01 -3.85211110e-01\n",
" 2.54215479e-01 -5.05547047e-01 -5.46367228e-01 3.39067727e-01\n",
" -2.22587734e-01 6.42588973e-01 3.65910411e-01 -4.12124813e-01\n",
" 4.58757669e-01 -9.60684597e-01 6.46253049e-01 -3.33079785e-01\n",
" -9.85556364e-01 -6.00200295e-01 -9.89162207e-01 7.13501215e-01\n",
" -1.55394569e-01 -2.30931655e-01 9.59562361e-01 1.40936166e-01\n",
" 3.53468925e-01 -5.29039614e-02 -7.52750099e-01 -1.00000000e+00\n",
" -5.59881687e-01 -2.51394302e-01 3.32646654e-03 -2.64209151e-01\n",
" -9.79841232e-01 -9.60728168e-01 6.28584266e-01 9.56566215e-01\n",
" 2.22542524e-01 9.99804378e-01 -2.51913786e-01 9.49084699e-01\n",
" -1.02457739e-01 -4.59917068e-01 2.09007233e-01 -5.08907616e-01\n",
" 7.47757733e-01 4.80907500e-01 -7.16885507e-01 1.65385246e-01\n",
" -3.98611091e-02 3.28421414e-01 -5.18680096e-01 -1.28824010e-01\n",
" -4.70307261e-01 -9.40705955e-01 -3.90891641e-01 9.33780849e-01\n",
" -1.80884019e-01 -7.91107059e-01 3.60995650e-01 -2.44044021e-01\n",
" -3.66987109e-01 8.48350883e-01 6.03457093e-01 3.85649234e-01\n",
" -2.97527373e-01 5.66402614e-01 3.45161766e-01 5.24850845e-01\n",
" -8.39007914e-01 1.28569022e-01 4.17470455e-01 -3.02416325e-01\n",
" -6.13402247e-01 -9.81145024e-01 -3.54902834e-01 4.99973416e-01\n",
" 9.89261806e-01 7.56967843e-01 3.58181804e-01 6.73808813e-01\n",
" -3.23846072e-01 6.81116641e-01 -9.60203171e-01 9.82460976e-01\n",
" -2.25113511e-01 3.24976534e-01 2.61794001e-01 2.99168348e-01\n",
" -8.29748273e-01 -1.74709514e-01 8.02439868e-01 -5.65778971e-01\n",
" -8.71301591e-01 -2.53270771e-02 -5.26067913e-01 -4.75921839e-01\n",
" -5.69297373e-01 5.55931330e-01 -3.41519982e-01 -3.95525753e-01\n",
" -1.25858366e-01 9.30840075e-01 9.57332134e-01 6.94689155e-01\n",
" -3.11512873e-02 6.69105232e-01 -9.11600351e-01 -5.16769290e-01\n",
" 1.17296688e-01 2.66171336e-01 2.33835682e-01 9.93997157e-01\n",
" -4.45861340e-01 -2.03810617e-01 -9.19092238e-01 -9.88888681e-01\n",
" -5.55789396e-02 -9.02743638e-01 -1.60378516e-01 -7.18693435e-01\n",
" 6.26380384e-01 9.41651613e-02 4.35321718e-01 4.65981126e-01\n",
" -9.89683211e-01 -7.76192129e-01 3.53938937e-01 -4.65756983e-01\n",
" 3.95055145e-01 -2.54643142e-01 6.81884885e-01 8.37408125e-01\n",
" -6.47483289e-01 6.29118919e-01 9.01026964e-01 -5.08751214e-01\n",
" -7.07996547e-01 8.01340103e-01 -3.15064162e-01 8.82692456e-01\n",
" -6.65321767e-01 9.87874150e-01 8.18870962e-01 7.96919882e-01\n",
" -9.24905121e-01 -5.41692376e-01 -8.46875191e-01 -5.50574780e-01\n",
" -1.36960626e-01 -1.99183986e-01 7.41399407e-01 5.99635839e-01\n",
" 4.43031877e-01 5.64272940e-01 -6.68283343e-01 9.96692359e-01\n",
" -5.52202284e-01 -9.57607627e-01 -2.02492654e-01 -3.18287194e-01\n",
" -9.88939285e-01 7.23998368e-01 3.34089667e-01 -1.15064375e-01\n",
" -5.06407976e-01 -5.43748260e-01 -9.62785900e-01 8.63424420e-01\n",
" 3.18572111e-02 9.83396888e-01 -2.96029709e-02 -9.26814198e-01\n",
" -5.78279555e-01 -9.34102118e-01 -2.01762229e-01 -2.35581771e-01\n",
" 4.78967540e-02 -1.89393640e-01 -9.34323311e-01 5.42665601e-01\n",
" 4.43590552e-01 5.13680518e-01 -6.03940070e-01 9.98171926e-01\n",
" 9.99999702e-01 9.74932194e-01 8.85953546e-01 9.10931945e-01\n",
" -9.98830616e-01 -4.37183648e-01 9.99993563e-01 -9.73267496e-01\n",
" -1.00000000e+00 -9.18928504e-01 -6.39754355e-01 2.06475183e-01\n",
" -1.00000000e+00 -2.00628757e-01 1.15270138e-01 -9.15450156e-01\n",
" 2.60408372e-01 9.80830848e-01 9.90411162e-01 -1.00000000e+00\n",
" 8.45194995e-01 9.19868171e-01 -6.63830996e-01 8.75674129e-01\n",
" -3.53310645e-01 9.69174147e-01 4.22582924e-01 4.03783411e-01\n",
" -3.32168251e-01 4.30539131e-01 -7.90726364e-01 -8.57651889e-01\n",
" -2.46795446e-01 -4.31535482e-01 9.88692582e-01 1.27706081e-01\n",
" -7.57868111e-01 -9.31355178e-01 1.94698200e-01 -1.07134461e-01\n",
" -1.97656348e-01 -9.61324692e-01 -1.54280528e-01 3.25440675e-01\n",
" 7.48623610e-01 1.88400581e-01 3.22433650e-01 -6.85797215e-01\n",
" 2.63898194e-01 -3.59514982e-01 3.32349211e-01 6.80141866e-01\n",
" -9.41358328e-01 -5.31072617e-01 -8.40196460e-02 -3.92503917e-01\n",
" -3.32295567e-01 -9.58871245e-01 9.69267845e-01 -4.26912040e-01\n",
" 7.04783678e-01 1.00000000e+00 -1.67649701e-01 -8.80904377e-01\n",
" 5.32298625e-01 2.54779428e-01 5.04027270e-02 1.00000000e+00\n",
" 7.80759752e-01 -9.83533561e-01 -5.50854862e-01 5.80364823e-01\n",
" -6.25309765e-01 -6.65773213e-01 9.99483824e-01 -2.84368336e-01\n",
" -4.23684865e-01 9.55960527e-02 9.82141495e-01 -9.91048336e-01\n",
" 9.76443887e-01 -9.28290844e-01 -9.66890872e-01 9.65992451e-01\n",
" 9.39904273e-01 -5.29012918e-01 -6.77458227e-01 1.37971729e-01\n",
" -5.10756612e-01 2.64738441e-01 -9.43566918e-01 6.76875114e-01\n",
" 3.42358977e-01 -9.48977843e-02 8.79352093e-01 -8.35172236e-01\n",
" -5.57096660e-01 4.17479962e-01 -3.69297534e-01 1.52817219e-01\n",
" 8.56687069e-01 5.89256048e-01 -2.54680037e-01 5.76275252e-02\n",
" -3.41867715e-01 -2.47040004e-01 -9.74508464e-01 3.31176996e-01\n",
" 1.00000000e+00 -8.08946416e-02 2.30953231e-01 -4.37440664e-01\n",
" -3.81645486e-02 -1.49804518e-01 4.91201341e-01 4.89298314e-01\n",
" -3.21012318e-01 -8.63301396e-01 5.81488132e-01 -9.47197855e-01\n",
" -9.88454282e-01 6.87534451e-01 1.52578264e-01 -4.47930098e-01\n",
" 9.99984980e-01 5.34064829e-01 2.19121948e-01 3.67227137e-01\n",
" 9.43343937e-01 2.88104219e-03 5.48833132e-01 5.84811509e-01\n",
" 9.75402236e-01 -1.80317044e-01 5.53062618e-01 8.11296344e-01\n",
" -7.42244005e-01 -3.87656391e-01 -6.70202076e-01 4.43656929e-02\n",
" -9.32610154e-01 -3.03909630e-02 -9.59662199e-01 9.61782217e-01\n",
" 8.26276779e-01 3.48599970e-01 2.50447959e-01 4.04596806e-01\n",
" 1.00000000e+00 -1.16176875e-02 6.40972257e-01 -2.00082302e-01\n",
" 8.12614202e-01 -9.98215735e-01 -8.05521309e-01 -4.15433884e-01\n",
" -9.32648592e-03 -5.55312932e-01 -3.62298608e-01 3.09281796e-01\n",
" -9.65382695e-01 5.58270812e-01 3.47898096e-01 -9.88508165e-01\n",
" -9.90992010e-01 1.75204128e-02 7.38597155e-01 5.21294586e-02\n",
" -9.43995655e-01 -7.00239480e-01 -6.14376664e-01 4.72313285e-01\n",
" -2.33542457e-01 -9.22834396e-01 2.87253596e-02 -3.23268652e-01\n",
" 5.22115707e-01 -1.93638489e-01 5.83559692e-01 6.65517628e-01\n",
" 7.54108548e-01 -4.18706387e-01 -2.26379082e-01 -1.10005960e-01\n",
" -8.09102535e-01 7.71164834e-01 -7.88796842e-01 -7.82582879e-01\n",
" -1.25516132e-01 1.00000000e+00 -4.62275952e-01 6.52186751e-01\n",
" 6.75179601e-01 6.98102713e-01 -1.89331502e-01 1.47414282e-01\n",
" 8.40114832e-01 2.86029547e-01 -4.61848646e-01 -4.46113944e-01\n",
" -4.91814762e-01 -3.87856096e-01 6.60984635e-01 3.29753235e-02\n",
" 4.26733404e-01 7.97589064e-01 6.98115766e-01 7.32698292e-02\n",
" -1.32251196e-02 1.03879705e-01 9.99356806e-01 -2.79265940e-01\n",
" -1.74580112e-01 -4.70604390e-01 -5.66857904e-02 -3.83076370e-01\n",
" -2.91456074e-01 1.00000000e+00 3.01932126e-01 5.39371520e-02\n",
" -9.91273701e-01 -6.93168938e-01 -9.03439462e-01 9.99997556e-01\n",
" 7.89496779e-01 -7.76895404e-01 6.63245499e-01 5.57200432e-01\n",
" -2.62795001e-01 7.13438749e-01 -1.92062631e-01 -2.67771125e-01\n",
" 2.93030471e-01 1.20144993e-01 9.52840984e-01 -5.37634909e-01\n",
" -9.72682714e-01 -6.06430411e-01 4.74803299e-01 -9.55877364e-01\n",
" 9.99399424e-01 -6.48434699e-01 -2.35546544e-01 -4.12093878e-01\n",
" -2.59444825e-02 2.27243394e-01 -3.50962393e-02 -9.81136203e-01\n",
" -2.68580884e-01 1.58754587e-01 9.69623446e-01 2.72614628e-01\n",
" -5.84368408e-01 -8.90106082e-01 5.79616010e-01 5.12818158e-01\n",
" -6.97850406e-01 -9.45521295e-01 9.69676375e-01 -9.81482327e-01\n",
" 5.80184579e-01 1.00000000e+00 3.74803931e-01 -3.16616893e-01\n",
" 2.52832353e-01 -4.28504556e-01 3.01271737e-01 -4.09856349e-01\n",
" 6.53815985e-01 -9.58217800e-01 -3.00675660e-01 -2.71157175e-01\n",
" 4.31750059e-01 -1.19413987e-01 -9.46294963e-02 7.10950017e-01\n",
" 2.44034618e-01 -5.69593549e-01 -6.55507863e-01 -9.31165367e-02\n",
" 3.97726655e-01 7.81391501e-01 -2.73002118e-01 -1.66758135e-01\n",
" 8.34571719e-02 -1.87023386e-01 -9.23197985e-01 -3.48046541e-01\n",
" -4.93540585e-01 -9.99940574e-01 5.97529531e-01 -1.00000000e+00\n",
" 2.50703275e-01 -1.31433690e-02 -2.36405879e-01 8.24405313e-01\n",
" 3.70994270e-01 3.30683202e-01 -7.12654889e-01 -5.37648737e-01\n",
" 6.52847707e-01 8.09146702e-01 -3.00231606e-01 -3.43623936e-01\n",
" -6.49569273e-01 3.00527364e-01 -1.92918777e-01 3.33070248e-01\n",
" -3.79284352e-01 7.63028145e-01 -1.91800609e-01 1.00000000e+00\n",
" 1.17010057e-01 -5.30115008e-01 -9.75999892e-01 2.45506227e-01\n",
" -3.15001935e-01 1.00000000e+00 -8.33743811e-01 -9.60250914e-01\n",
" 3.80856931e-01 -7.05618262e-01 -7.84226358e-01 3.14483225e-01\n",
" 5.31871468e-02 -8.12366188e-01 -8.68545413e-01 9.44719493e-01\n",
" 8.33881915e-01 -5.63703418e-01 4.01934952e-01 -3.12713683e-01\n",
" -5.38256884e-01 1.22065157e-01 6.71774387e-01 9.89326298e-01\n",
" 1.81289807e-01 8.66247773e-01 2.89487422e-01 -1.72825232e-01\n",
" 9.66990769e-01 3.03174257e-01 3.69202971e-01 2.21820295e-01\n",
" 1.00000000e+00 3.18685412e-01 -9.35128689e-01 2.45870575e-01\n",
" -9.79029357e-01 -3.01807880e-01 -9.63689148e-01 3.52392346e-01\n",
" 2.29213819e-01 8.62274766e-01 -2.90317357e-01 9.58398342e-01\n",
" -4.79456872e-01 1.12460546e-01 -4.52855527e-01 -6.65019006e-02\n",
" 4.28122997e-01 -8.92569840e-01 -9.86009181e-01 -9.84905064e-01\n",
" 5.65347910e-01 -4.62038249e-01 -1.21679425e-01 2.58967400e-01\n",
" 1.59082562e-01 4.09519970e-01 4.97945547e-01 -1.00000000e+00\n",
" 9.34118390e-01 4.31054741e-01 7.58154869e-01 9.62230623e-01\n",
" 6.01305366e-01 4.47203606e-01 2.33327925e-01 -9.82532561e-01\n",
" -9.76686537e-01 -4.01512414e-01 -2.55510837e-01 7.07738161e-01\n",
" 6.64976895e-01 8.65667641e-01 4.77845669e-01 -4.98781234e-01\n",
" -2.12694705e-01 -7.88345933e-02 -5.50440431e-01 -9.93793011e-01\n",
" 5.43924034e-01 -3.48899305e-01 -9.57004845e-01 9.60863113e-01\n",
" -1.37568846e-01 -1.32730260e-01 1.43163621e-01 -5.77764392e-01\n",
" 9.20214176e-01 7.48068333e-01 4.04670805e-01 3.60171832e-02\n",
" 4.78334367e-01 8.90343130e-01 9.50072825e-01 9.86242473e-01\n",
" -6.64497674e-01 7.80569971e-01 -3.45106184e-01 5.04795969e-01\n",
" 4.85483915e-01 -9.41858947e-01 1.54408753e-01 3.52799833e-01\n",
" -4.82338339e-01 3.10353547e-01 -3.31817776e-01 -9.54289675e-01\n",
" 5.15970707e-01 -3.45210344e-01 5.34975588e-01 -4.60633814e-01\n",
" 8.68272781e-02 -4.87649977e-01 -1.79318368e-01 -7.03506172e-01\n",
" -6.69845700e-01 6.88161969e-01 4.41581219e-01 9.00379241e-01\n",
" 7.40806401e-01 -1.24433160e-01 -7.29168415e-01 -2.91805655e-01\n",
" -5.07772446e-01 -9.17512000e-01 9.02978718e-01 -1.98445059e-02\n",
" -6.61094710e-02 2.64029860e-01 -1.04134716e-01 6.04383647e-01\n",
" -5.06777354e-02 -4.30096388e-01 -3.38814467e-01 -7.48992860e-01\n",
" 8.48512590e-01 -2.51320213e-01 -5.50926447e-01 -4.59069490e-01\n",
" 6.79095089e-01 2.88966238e-01 9.99845088e-01 -5.57984591e-01\n",
" -7.43643522e-01 -3.74783367e-01 -3.49461287e-01 4.05343294e-01\n",
" -4.87777978e-01 -1.00000000e+00 3.66959989e-01 -3.45835388e-01\n",
" 4.91553366e-01 -4.42789882e-01 6.90753639e-01 -3.93862993e-01\n",
" -9.77467537e-01 -2.70980477e-01 3.86985451e-01 4.09851670e-01\n",
" -4.88349944e-01 -5.71954727e-01 5.28122962e-01 -5.58347255e-02\n",
" 9.09353554e-01 8.76048863e-01 -3.21841799e-02 2.95268565e-01\n",
" 6.29843771e-01 -2.49958694e-01 -7.40591824e-01 9.05128181e-01]], shape=(1, 768), dtype=float32)\n"
]
}
],
"source": [
"import os\n",
"os.environ['TF_USE_LEGACY_KERAS'] = '1'\n",
"import tensorflow as tf\n",
"import tensorflow_hub as hub\n",
"import tensorflow_text as text\n",
"text_input = tf.keras.layers.Input(shape=(), dtype=tf.string)\n",
"preprocessor = hub.KerasLayer(\n",
" \"https://kaggle.com/models/tensorflow/bert/TensorFlow2/en-uncased-preprocess/3\")\n",
"encoder_inputs = preprocessor(text_input)\n",
"encoder = hub.KerasLayer(\n",
" \"https://kaggle.com/models/tensorflow/bert/TensorFlow2/en-uncased-l-12-h-768-a-12/3\",\n",
" trainable=True)\n",
"outputs = encoder(encoder_inputs)\n",
"pooled_output = outputs[\"pooled_output\"] # [batch_size, 768].\n",
"sequence_output = outputs[\"sequence_output\"] # [batch_size, seq_length, 768].\n",
"embedding_model = tf.keras.Model(text_input, pooled_output)\n",
"sentences = tf.constant([\"(your text here)\"])\n",
"print(embedding_model(sentences))\n"
]
},
{
"cell_type": "markdown",
"id": "73363774-85b0-4cdf-b0a7-8bb937061577",
"metadata": {},
"source": [
"### 2.3 图像模型检查\n",
"预期结果：\n",
"1. 程序正常运行；\n",
"2. accuracy达到0.9以上。"
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "38395dd8-b4b9-4a68-813c-e24344771af9",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-12 15:25:24.283485: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.\n",
"To enable the following instructions: SSE3 SSE4.1 SSE4.2 AVX AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.\n",
"2024-10-12 15:25:26.781495: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1639] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 63225 MB memory: -> device: 0, name: DCU K100_AI, pci bus id: 0000:16:00.0\n",
"2024-10-12 15:25:27.613083: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.020912: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.022034: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.031351: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.032331: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.037432: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.050056: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.052597: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.053980: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.055220: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.056439: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.057287: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.059033: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.061178: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.061945: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.071138: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.071948: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.073080: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.073806: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.078874: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.079613: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.774022: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.777030: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.818869: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.827270: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.828499: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.835784: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.837421: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-12 15:25:29.975935: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.978086: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.986188: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.987431: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.988496: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.989635: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.990639: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:29.991438: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:30.004742: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:30.006526: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:30.007302: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:30.047402: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:30.054007: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:30.058284: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:30.405201: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:38.306620: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7eebf3408580 initialized for platform ROCM (this does not guarantee that XLA will be used). Devices:\n",
"2024-10-12 15:25:38.306664: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): DCU K100_AI, DCU ISA version: gfx928:sramecc+:xnack-\n",
"2024-10-12 15:25:38.313714: I tensorflow/compiler/mlir/tensorflow/utils/dump_mlir_util.cc:255] disabling MLIR crash reproducer, set env var `MLIR_CRASH_REPRODUCER_DIRECTORY` to enable.\n",
"2024-10-12 15:25:39.208733: I ./tensorflow/compiler/jit/device_compiler.h:186] Compiled cluster using XLA! This line is logged at most once for the lifetime of the process.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"1875/1875 [==============================] - 14s 2ms/step - loss: 0.2920 - accuracy: 0.9159\n",
"Epoch 2/5\n",
" 77/1875 [>.............................] - ETA: 3s - loss: 0.1604 - accuracy: 0.9541"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"2024-10-12 15:25:44.507681: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:44.510552: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:44.511623: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n",
"2024-10-12 15:25:44.515506: I tensorflow/core/common_runtime/gpu_fusion_pass.cc:508] ROCm Fusion is enabled.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"1875/1875 [==============================] - 4s 2ms/step - loss: 0.1407 - accuracy: 0.9585\n",
"Epoch 3/5\n",
"1875/1875 [==============================] - 4s 2ms/step - loss: 0.1064 - accuracy: 0.9678\n",
"Epoch 4/5\n",
"1875/1875 [==============================] - 4s 2ms/step - loss: 0.0857 - accuracy: 0.9742\n",
"Epoch 5/5\n",
"1875/1875 [==============================] - 4s 2ms/step - loss: 0.0734 - accuracy: 0.9771\n"
]
},
{
"data": {
"text/plain": [
"<keras.src.callbacks.History at 0x7f0c246fd100>"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import tensorflow as tf\n",
"mnist = tf.keras.datasets.mnist\n",
"\n",
"(x_train, y_train),(x_test, y_test) = mnist.load_data()\n",
"x_train, x_test = x_train / 255.0, x_test / 255.0\n",
"\n",
"model = tf.keras.models.Sequential([\n",
" tf.keras.layers.Flatten(input_shape=(28, 28)),\n",
" tf.keras.layers.Dense(128, activation='relu'),\n",
" tf.keras.layers.Dropout(0.2),\n",
" tf.keras.layers.Dense(10, activation='softmax')\n",
"])\n",
"\n",
"model.compile(optimizer='adam',\n",
" loss='sparse_categorical_crossentropy',\n",
" metrics=['accuracy'])\n",
"\n",
"model.fit(x_train, y_train, epochs=5)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9223831e-ee21-47c0-a7ef-220a02763d30",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.20"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment