Skip to content

GitLab

Commit c873301f authored by wanglch's avatar wanglch
Browse files

Initial commit

parents
Hide whitespace changes
Inline Side-by-side
Showing with 3886 additions and 0 deletions
Contributors.md 0 → 100644
View file @ c873301f
## Code of Conduct 🤝
Before diving in, please take a moment to review our Code of Conduct. It sets the tone for our community and emphasizes the importance of respect and inclusivity. [Read the Code of Conduct](LICENSE.md).
## Contribution Types 🦠🚀📚
### Bug Reports 🐞
If you encounter any bugs during your journey, don't fret! We have the Bug Busters ready to help. To report a bug, follow these steps:
1. Check if the bug has already been reported in [GitHub Issues](https://github.com/AI4Finance-Foundation/FinGPT/issues).
2. If it's a new bug, open a new issue with a concise description and provide detailed, step-by-step instructions to reproduce it.
### Feature Requests 💡
Do you have visionary ideas that could elevate FinGPT? Share them with us! When submitting a feature request, be sure to include:
1. A clear and vivid description of the feature you envision.
2. Discuss the impact and potential benefits.
### Documentation 📖
For those with a penchant for words and an eye for detail, consider contributing to our documentation. You can make the documentation more enlightening for everyone. 🧙📜
### Code Contributions 💻
Calling all AI heroes and wizards! You are the secret sauce behind the FinGPT project. To contribute code and save the financial world:
1. **Fork the Repository**: Click the "Fork" button on the top right of the repository's page. This creates your own copy of the project.
2. **Clone your Fork**: In your terminal, use the following command to clone your fork to your local machine:
```bash
git clone https://github.com/YourUsername/FinGPT.git
```
3. **Create a New Branch**: Make a new branch for your adventures. This helps keep the main codebase clean:
```bash
git checkout -b your-feature-branch
```
4. **Work Your Magic**: Implement your code or changes.
5. **Commit and Push**: Use these commands to commit your changes and push them to your fork:
```bash
git commit -m "Your commit message"
git push origin your-feature-branch
```
6. **Create a Pull Request**: Go to the original FinGPT repository and click "New Pull Request." Select your branch, write a description, and submit.
## Seeking Assistance ❓🙋‍♀️
If you find yourself stuck or have questions, remember that our support team is your sidekick. Don't hesitate to reach out. We are here to guide you through the process and provide any necessary assistance.
## Getting Started 🚀🚀
Are you ready to make a mark on the FinGPT project? Grab your cape and join us in our mission to make finance and AI even more incredible. Your contributions are the magic that fuels our journey.
🔗 [FinGPT GitHub Repository](https://github.com/AI4Finance-Foundation/FinGPT)
### May your contributions be as amazing as you are! 🌌🚀
\ No newline at end of file
LICENSE 0 → 100644
View file @ c873301f
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
\ No newline at end of file
README.md 0 → 100644
View file @ c873301f
# Vary-toy:
**免费开源多模态OCR大模型**
## 论文
- [论文地址][Small Language Model Meets with Reinforced Vision Vocabulary](https://arxiv.org/abs/2401.12503)
- [Vary-toy weights huggingface 预训练模型下载地址]<https://huggingface.co/Haoran-megvii/Vary-toy>
- [Download the CLIP-VIT-L]<https://huggingface.co/openai/clip-vit-large-patch14/>
## 模型结构
### Vary-toy模型结构
<div align="center">
<img align="center" src=image/model.png>
</div>
## 算法原理
Vary-toy 利用 Vary-tiny+ 管道来生成新的愿景 Vary-toy的词汇。这样的视觉词汇可以有效地编码密集的文本和自然物体位置信息转换为令牌。基于改进的词汇量，Vary-toy 不仅拥有所有以前的功能（文档OCR），但也很好地处理了对象检测任务。
## 环境配置
### Docker（方法一）
注：在部署环境前需将Vary-toy/vary/demo/run_qwen_vary.py和Vary-toy/vary/model/vary_qwen_vary.py和Vary-toy/vary/model/vary_toy_qwen1_8.py中的模型路径改为本地模型路径，同时将Vary-toy/cache/models--HaoranWei--Vary-toy模型中的config.json文件中的模型路径改为本地路径，完成以上操作后再执行pip install e .指令。
```
docker pull image.sourcefind.cn:5000/dcu/admin/base/pytorch:2.1.0-ubuntu22.04-dtk23.10.1-py310
docker run -it -v /path/your_code_data/:/path/your_code_data/ -v /opt/hyhal/:/opt/hyhal/:ro --shm-size=64G --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video --name vary-toy <your imageID> bash
docker exec -it vary-toy bash
cd /path/your_code_data/vary-toy
pip install e .
pip install ninja
```
### Dockerfile（方法二）
```
cd /path/your_code_data/vary-toy/docker
docker build --no-cache -t vary-toy:latest .
docker run --shm-size=64G --name vary-toy -v /opt/hyhal:/opt/hyhal:ro --privileged=true --device=/dev/kfd --device=/dev/dri/ --group-add video -v /path/your_code_data/:/path/your_code_data/ -it vary-toy bash
```
### Anaconda（方法三）
关于本项目DCU显卡所需的特殊深度学习库可从[光合](https://developer.hpccube.com/tool/)开发者社区下载安装。
```
DTK驱动：dtk23.10
python：python3.10
torch:2.1
torchvision: 0.16.0
deepspped: 0.12.3
```
```
conda create -n vary-toy python=3.10
conda activate vary-toy
cd /path/your_code_data/vary-toy
pip install e .
pip install ninja
```
`Tips：以上dtk驱动、python、torch、deepspeed等DCU相关工具版本需要严格一一对应`
## 数据集
无, 暂未开放数据集
## 训练
需自己构建数据集
```
deepspeed Vary/train/train_qwen_vary.py --deepspeed /Vary/zero_config/zero2.json
--model_name_or_path /Vary-toy/path/
--vision_tower /vit-large-patch14/path/
--freeze_vision_tower True
--freeze_lm_model False
--vision_select_layer -2
--use_im_start_end True
--bf16 True
--per_device_eval_batch_size 4
--gradient_accumulation_steps 1
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 5000
--save_total_limit 1
--weight_decay 0.
--warmup_ratio 0.03
--lr_scheduler_type "cosine"
--logging_steps 1 --tf32 True
--model_max_length 4096
--gradient_checkpointing True
--dataloader_num_workers 4
--report_to none
--per_device_train_batch_size 4
--num_train_epochs 1
--learning_rate 5e-5
--datasets data_name1+data_name2+data_name3
--output_dir /path/to/output/
```
## 推理
**需严格按照本仓库代码目录进行排列**
备注：修改 --image-file 替换ocr文件
```
python /home/wanglch/projects/Vary-toy/vary/demo/run_qwen_vary.py --model-name /home/wanglch/projects/Vary-toy/cache/models--HaoranWei--Vary-toy --image-file /home/wanglch/projects/Vary-toy/image/pic.jpg
```
备注：修改 vary/demo/run_qwen_vary.py 替换57行代码执行不同任务操作
```
qs = 'Provide the ocr results of this image.' # 执行ocr任务
qs = 'Detevate the ** in this image.' # 检测任务
qs = 'Convert the document to markdown format.' # 公式转markdown
qs = 'Describe this image in within 100 words.' # 多模态描述
```
### 推理代码
```
bash run.sh
```
## result
### 英语文档ocr
<div align="center">
<img align="center" src=image/ocr_en.png>
</div>
### 中文文档ocr
<div align="center">
<img align="center" src=image/ocr_en.png>
</div>
### 车牌识别ocr
<div align="center">
<img align="center" src=image/car.png>
</div>
<div align="center">
<img align="center" src=image/car_number.png>
</div>
### 精度
无
## 应用场景
金融,教育,政府,科研,交通,医疗
### 算法类别
图文OCR
## 预训练权重
- Vary-toy 权重下载 [here](https://huggingface.co/Haoran-megvii/Vary-toy).
- CLIP-VIT-L 权重下载 [here](https://huggingface.co/openai/clip-vit-large-patch14/).
## 参考资料
- 本项目gitlab地址[Ucas-HaoranWei/Vary-toy](https://github.com/Ucas-HaoranWei/Vary-toy/tree/main)
README_toy.md 0 → 100644
View file @ c873301f
<h3><a href="">Small Language Model Meets with Reinforced Vision Vocabulary</a></h3>
<a href="https://varytoy.github.io/"><img src="https://img.shields.io/badge/Project-Page-Green"></a>
<a href="https://arxiv.org/abs/2401.12503"><img src="https://img.shields.io/badge/Paper-PDF-orange"></a>
<a href="https://vary.xiaomy.net/"><img src="https://img.shields.io/badge/demo-blue"></a>
<a href="https://zhuanlan.zhihu.com/p/679447793"><img src="https://img.shields.io/badge/zhihu-yellow"></a>
[Haoran Wei*](https://scholar.google.com/citations?user=J4naK0MAAAAJ&hl=en), Lingyu Kong*, Jinyue Chen, Liang Zhao, [Zheng Ge](https://joker316701882.github.io/), [En Yu](https://scholar.google.com.hk/citations?user=rWCQMNgAAAAJ&hl=zh-CN&oi=sra), [Jianjian Sun](https://scholar.google.com/citations?user=MVZrGkYAAAAJ&hl=en), Chunrui Han, [Xiangyu Zhang](https://scholar.google.com/citations?user=yuB-cfoAAAAJ&hl=en)
<p align="center">
<img src="assets/vary-toy-logo.jpg" style="width: 200px" align=center>
</p>
<p align="center">
<a href="">The Young's First ``Large'' Vision Language Model</a>
</p>
## Release
- [2024/4/21] 🔥🔥🔥 For OneChart, we have released the web demo in [Project Page](https://onechartt.github.io/). Have fun!!
- [2024/4/21] 🔥🔥🔥 We present a Vary-tiny LAVIS codebase (for training from scratch) and the Vary-600k dataset (300K English and 300K Chinese pages) [here](https://github.com/Ucas-HaoranWei/Vary-tiny-600k) !!!
- [2024/4/15]🔥🔥🔥We release a chart parsing model OneChart [here](https://github.com/LingyvKong/OneChart).
- [2024/4/12]🔥🔥🔥We will release a chart parsing model based on Vary-tiny next week. The model supports both English and Chinese charts.
- [2024/3/16]🔥🔥🔥I found many friends very interested in Vary-tiny(OPT-125M), so I opened source it [here](https://huggingface.co/HaoranWei/Vary-tiny-opt125M/tree/main), a PDF-dense OCR and object detection version.
- [2024/1/23] 🔥Eval codes will be available soon.
- [2024/1/23] 🔥🔥🔥You only need a single 1080Ti to experience all features of current LVLMs.
[![Code License](https://img.shields.io/badge/Code%20License-Apache_2.0-green.svg)](https://github.com/tatsu-lab/stanford_alpaca/blob/main/LICENSE)
[![Data License](https://img.shields.io/badge/Data%20License-CC%20By%20NC%204.0-red.svg)](https://github.com/tatsu-lab/stanford_alpaca/blob/main/DATA_LICENSE)
**Usage and License Notices**: The data, code, and checkpoint are intended and licensed for research use only. They are also restricted to use that follow the license agreement of LLaMA, Vicuna, GPT-4, Qwen, and LLaVA.
## Contents
- [Install](#install)
- [Vary-toy Weights](#vary-weights)
- [Demo](#Demo)
- [Train](#train)
## Note
If you have built the original [Vary](https://github.com/Ucas-HaoranWei/Vary), please rebuild this repo !!!
## Install
1. Clone this repository and navigate to the Vary folder
```bash
git clone https://github.com/Ucas-HaoranWei/Vary-toy.git
cd /path/to/vary-toy
```
2. Install Package
```Shell
conda create -n vary python=3.10 -y
conda activate vary
pip install e .
```
3. Install Flash-Attention
```
pip install ninja
pip install flash-attn --no-build-isolation
```
## Vary Weights
- Download the Vary-toy weights [here](https://huggingface.co/Haoran-megvii/Vary-toy).
- Download the CLIP-VIT-L [here](https://huggingface.co/openai/clip-vit-large-patch14/).
## Demo
1. Update the CLIP-VIT path in the codes (/cache/vit-large-patch14/) to your path.
2.
```Shell
python vary/demo/run_qwen_vary.py --model-name /vary/model/path/ --image-file /an/image/file.png
```
## Train
```Shell
deepspeed Vary/train/train_qwen_vary.py --deepspeed /Vary/zero_config/zero2.json
--model_name_or_path /Vary-toy/path/
--vision_tower /vit-large-patch14/path/
--freeze_vision_tower True
--freeze_lm_model False
--vision_select_layer -2
--use_im_start_end True
--bf16 True
--per_device_eval_batch_size 4
--gradient_accumulation_steps 1
--evaluation_strategy "no"
--save_strategy "steps"
--save_steps 5000
--save_total_limit 1
--weight_decay 0.
--warmup_ratio 0.03
--lr_scheduler_type "cosine"
--logging_steps 1 --tf32 True
--model_max_length 4096
--gradient_checkpointing True
--dataloader_num_workers 4
--report_to none
--per_device_train_batch_size 4
--num_train_epochs 1
--learning_rate 5e-5
--datasets data_name1+data_name2+data_name3
--output_dir /path/to/output/
```
We encourage you to extract the new vision vocabulary weights for your new base language model !!!
## Contact
If you have any questions about the code or the paper, please email (`weihaoran18@mails.ucas.ac.cn`).
## Discussion
Vary-toy is not a toy, and we have designed two excellent models based on it, one is Vary-document (specifically for document/pdf processing), and the other is Vary-plot for chart analysis. You can see their amazing performance here [Vary-family](https://github.com/Ucas-HaoranWei/Vary-family).
## Citation
If you find our work useful in your research, please consider citing Vary:
```bibtex
@article{wei2023vary,
title={Vary: Scaling up the Vision Vocabulary for Large Vision-Language Models},
author={Wei, Haoran and Kong, Lingyu and Chen, Jinyue and Zhao, Liang and Ge, Zheng and Yang, Jinrong and Sun, Jianjian and Han, Chunrui and Zhang, Xiangyu},
journal={arXiv preprint arXiv:2312.06109},
year={2023}
}
@article{wei2024small,
title={Small Language Model Meets with Reinforced Vision Vocabulary},
author={Wei, Haoran and Kong, Lingyu and Chen, Jinyue and Zhao, Liang and Ge, Zheng and Yu, En and Sun, Jianjian and Han, Chunrui and Zhang, Xiangyu},
journal={arXiv preprint arXiv:2401.12503},
year={2024}
}
```
cache/models--HaoranWei--Vary-toy/.gitattributes 0 → 100644
View file @ c873301f
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ckpt filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.mlmodel filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.npy filter=lfs diff=lfs merge=lfs -text
*.npz filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pickle filter=lfs diff=lfs merge=lfs -text
*.pkl filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
*.safetensors filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tar filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.wasm filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zst filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
cache/models--HaoranWei--Vary-toy/README.md 0 → 100644
View file @ c873301f
![image/png](https://cdn-uploads.huggingface.co/production/uploads/6436618aeef1f55654a9f458/b82U8BLdTqEWxZiP3eVQJ.png)
cache/models--HaoranWei--Vary-toy/config.json 0 → 100644
View file @ c873301f
{
"_name_or_path": "/home/wanglch/projects/Vary-toy/cache/models--HaoranWei--Vary-toy",
"architectures": [
"MMGPTQwenForCausalLM"
],
"attn_dropout_prob": 0.0,
"auto_map": {
"AutoConfig": "configuration_qwen.QWenConfig",
"AutoModelForCausalLM": "modeling_qwen.QWenLMHeadModel"
},
"bf16": false,
"emb_dropout_prob": 0.0,
"fp16": false,
"fp32": false,
"freeze_vision_tower": false,
"hidden_size": 2048,
"im_end_token": 151858,
"im_patch_token": 151859,
"im_start_token": 151857,
"image_token_len": 256,
"initializer_range": 0.02,
"intermediate_size": 11008,
"kv_channels": 128,
"layer_norm_epsilon": 1e-06,
"max_position_embeddings": 8192,
"model_type": "mmgpt",
"no_bias": true,
"num_attention_heads": 16,
"num_hidden_layers": 24,
"onnx_safe": null,
"rotary_emb_base": 10000,
"rotary_pct": 1.0,
"scale_attn_weights": true,
"seq_length": 8192,
"tie_word_embeddings": false,
"tokenizer_class": "QWenTokenizer",
"torch_dtype": "bfloat16",
"transformers_version": "4.32.1",
"use_cache": true,
"use_dynamic_ntk": true,
"use_flash_attn": "auto",
"use_im_start_end": true,
"use_logn_attn": true,
"vision_select_layer": -2,
"vision_tower": "/mnt/host0/vit-large-patch14",
"vocab_size": 151860
}
cache/models--HaoranWei--Vary-toy/generation_config.json 0 → 100644
View file @ c873301f
{
"chat_format": "chatml",
"do_sample": true,
"eos_token_id": 151643,
"max_new_tokens": 512,
"max_window_size": 6144,
"pad_token_id": 151643,
"repetition_penalty": 1.1,
"top_k": 0,
"top_p": 0.8,
"transformers_version": "4.32.1"
}
cache/models--HaoranWei--Vary-toy/qwen.tiktoken 0 → 100644
View file @ c873301f
This source diff could not be displayed because it is too large. You can view the blob instead.
cache/models--HaoranWei--Vary-toy/tokenization_qwen.py 0 → 100644
View file @ c873301f
# Copyright (c) Alibaba Cloud.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.
"""Tokenization classes for QWen."""
import base64
import logging
import os
import unicodedata
from typing import Collection, Dict, List, Set, Tuple, Union
import tiktoken
from transformers import PreTrainedTokenizer, AddedToken
logger = logging.getLogger(__name__)
VOCAB_FILES_NAMES = {"vocab_file": "qwen.tiktoken"}
PAT_STR = r"""(?i:'s|'t|'re|'ve|'m|'ll|'d)|[^\r\n\p{L}\p{N}]?\p{L}+|\p{N}| ?[^\s\p{L}\p{N}]+[\r\n]*|\s*[\r\n]+|\s+(?!\S)|\s+"""
ENDOFTEXT = "<|endoftext|>"
IMSTART = "<|im_start|>"
IMEND = "<|im_end|>"
# as the default behavior is changed to allow special tokens in
# regular texts, the surface forms of special tokens need to be
# as different as possible to minimize the impact
EXTRAS = tuple((f"<|extra_{i}|>" for i in range(205)))
SPECIAL_TOKENS = (
ENDOFTEXT,
IMSTART,
IMEND,
) + EXTRAS
def _load_tiktoken_bpe(tiktoken_bpe_file: str) -> Dict[bytes, int]:
with open(tiktoken_bpe_file, "rb") as f:
contents = f.read()
return {
base64.b64decode(token): int(rank)
for token, rank in (line.split() for line in contents.splitlines() if line)
}
class QWenTokenizer(PreTrainedTokenizer):
"""QWen tokenizer."""
vocab_files_names = VOCAB_FILES_NAMES
def __init__(
self,
vocab_file,
errors="replace",
image_start_tag='<img>',
image_end_tag='</img>',
image_pad_tag='<imgpad>',
ref_start_tag='<ref>',
ref_end_tag='</ref>',
box_start_tag='<box>',
box_end_tag='</box>',
quad_start_tag='<quad>',
quad_end_tag='</quad>',
**kwargs,
):
super().__init__(**kwargs)
self.image_start_tag = image_start_tag
self.image_end_tag = image_end_tag
self.image_pad_tag = image_pad_tag
self.ref_start_tag = ref_start_tag
self.ref_end_tag = ref_end_tag
self.box_start_tag = box_start_tag
self.box_end_tag = box_end_tag
self.quad_start_tag = quad_start_tag
self.quad_end_tag = quad_end_tag
self.IMAGE_ST = (
ref_start_tag, ref_end_tag,
box_start_tag, box_end_tag,
quad_start_tag, quad_end_tag,
image_start_tag, image_end_tag,
image_pad_tag
)
self.errors = errors # how to handle errors in decoding
self.mergeable_ranks = _load_tiktoken_bpe(vocab_file) # type: dict[bytes, int]
self.special_tokens = {
token: index
for index, token in enumerate(
SPECIAL_TOKENS + self.IMAGE_ST, start=len(self.mergeable_ranks)
)
}
self.img_start_id = self.special_tokens[self.image_start_tag]
self.img_end_id = self.special_tokens[self.image_end_tag]
self.img_pad_id = self.special_tokens[self.image_pad_tag]
self.ref_start_id = self.special_tokens[self.ref_start_tag]
self.ref_end_id = self.special_tokens[self.ref_end_tag]
self.box_start_id = self.special_tokens[self.box_start_tag]
self.box_end_id = self.special_tokens[self.box_end_tag]
self.quad_start_id = self.special_tokens[self.quad_start_tag]
self.quad_end_id = self.special_tokens[self.quad_end_tag]
enc = tiktoken.Encoding(
"Qwen",
pat_str=PAT_STR,
mergeable_ranks=self.mergeable_ranks,
special_tokens=self.special_tokens,
)
assert (
len(self.mergeable_ranks) + len(self.special_tokens) == enc.n_vocab
), f"{len(self.mergeable_ranks) + len(self.special_tokens)} != {enc.n_vocab} in encoding"
self.decoder = {
v: k for k, v in self.mergeable_ranks.items()
} # type: dict[int, bytes|str]
self.decoder.update({v: k for k, v in self.special_tokens.items()})
self.tokenizer = enc # type: tiktoken.Encoding
self.eod_id = self.tokenizer.eot_token
self.im_start_id = self.special_tokens[IMSTART]
self.im_end_id = self.special_tokens[IMEND]
def __len__(self) -> int:
return self.tokenizer.n_vocab
def get_vocab(self) -> Dict[bytes, int]:
return self.mergeable_ranks
def convert_tokens_to_ids(
self, tokens: Union[bytes, str, List[Union[bytes, str]]]
) -> List[int]:
ids = []
if isinstance(tokens, (str, bytes)):
if tokens in self.special_tokens:
return self.special_tokens[tokens]
else:
return self.mergeable_ranks.get(tokens)
for token in tokens:
if token in self.special_tokens:
ids.append(self.special_tokens[token])
else:
ids.append(self.mergeable_ranks.get(token))
return ids
def _add_tokens(self, new_tokens: Union[List[str], List[AddedToken]], special_tokens: bool = False) -> int:
if not special_tokens and new_tokens:
raise ValueError('Adding regular tokens is not supported')
for token in new_tokens:
surface_form = token.content if isinstance(token, AddedToken) else token
if surface_form not in SPECIAL_TOKENS:
raise ValueError('Adding unknown special tokens is not supported')
return 0
def save_vocabulary(self, save_directory: str, **kwargs) -> Tuple[str]:
"""
Save only the vocabulary of the tokenizer (vocabulary).
Returns:
`Tuple(str)`: Paths to the files saved.
"""
file_path = os.path.join(save_directory, "qwen.tiktoken")
with open(file_path, "w", encoding="utf8") as w:
for k, v in self.mergeable_ranks.items():
line = base64.b64encode(k).decode("utf8") + " " + str(v) + "\n"
w.write(line)
return (file_path,)
def tokenize(
self,
text: str,
allowed_special: Union[Set, str] = "all",
disallowed_special: Union[Collection, str] = (),
**kwargs,
) -> List[Union[bytes, str]]:
"""
Converts a string in a sequence of tokens.
Args:
text (`str`):
The sequence to be encoded.
allowed_special (`Literal["all"]` or `set`):
The surface forms of the tokens to be encoded as special tokens in regular texts.
Default to "all".
disallowed_special (`Literal["all"]` or `Collection`):
The surface forms of the tokens that should not be in regular texts and trigger errors.
Default to an empty tuple.
kwargs (additional keyword arguments, *optional*):
Will be passed to the underlying model specific encode method.
Returns:
`List[bytes|str]`: The list of tokens.
"""
tokens = []
text = unicodedata.normalize("NFC", text)
# this implementation takes a detour: text -> token id -> token surface forms
for t in self.tokenizer.encode(
text, allowed_special=allowed_special, disallowed_special=disallowed_special
):
tokens.append(self.decoder[t])
return tokens
def convert_tokens_to_string(self, tokens: List[Union[bytes, str]]) -> str:
"""
Converts a sequence of tokens in a single string.
"""
text = ""
temp = b""
for t in tokens:
if isinstance(t, str):
if temp:
text += temp.decode("utf-8", errors=self.errors)
temp = b""
text += t
elif isinstance(t, bytes):
temp += t
else:
raise TypeError("token should only be of type types or str")
if temp:
text += temp.decode("utf-8", errors=self.errors)
return text
@property
def vocab_size(self):
return self.tokenizer.n_vocab
def _convert_id_to_token(self, index: int) -> Union[bytes, str]:
"""Converts an id to a token, special tokens included"""
if index in self.decoder:
return self.decoder[index]
raise ValueError("unknown ids")
def _convert_token_to_id(self, token: Union[bytes, str]) -> int:
"""Converts a token to an id using the vocab, special tokens included"""
if token in self.special_tokens:
return self.special_tokens[token]
if token in self.mergeable_ranks:
return self.mergeable_ranks[token]
raise ValueError("unknown token")
def _tokenize(self, text: str, **kwargs):
"""
Converts a string in a sequence of tokens (string), using the tokenizer. Split in words for word-based
vocabulary or sub-words for sub-word-based vocabularies (BPE/SentencePieces/WordPieces).
Do NOT take care of added tokens.
"""
raise NotImplementedError
def _decode(
self,
token_ids: Union[int, List[int]],
skip_special_tokens: bool = False,
errors: str = None,
**kwargs,
) -> str:
if isinstance(token_ids, int):
token_ids = [token_ids]
if skip_special_tokens:
token_ids = [i for i in token_ids if i < self.eod_id]
return self.tokenizer.decode(token_ids, errors=errors or self.errors)
cache/models--HaoranWei--Vary-toy/tokenizer_config.json 0 → 100644
View file @ c873301f
{
"auto_map": {
"AutoTokenizer": [
"tokenization_qwen.QWenTokenizer",
null
]
},
"clean_up_tokenization_spaces": true,
"model_max_length": 4096,
"padding_side": "right",
"tokenizer_class": "QWenTokenizer"
}
cache/models--HaoranWei--Vary-toy/trainer_state.json 0 → 100644
View file @ c873301f
{
"best_metric": null,
"best_model_checkpoint": null,
"epoch": 1.0,
"eval_steps": 500,
"global_step": 426,
"is_hyper_param_search": false,
"is_local_process_zero": true,
"is_world_process_zero": true,
"log_history": [
{
"epoch": 0.0,
"learning_rate": 1.5384615384615387e-06,
"loss": 0.92,
"step": 1
},
{
"epoch": 0.0,
"learning_rate": 3.0769230769230774e-06,
"loss": 0.9533,
"step": 2
},
{
"epoch": 0.01,
"learning_rate": 4.615384615384616e-06,
"loss": 0.9873,
"step": 3
},
{
"epoch": 0.01,
"learning_rate": 6.153846153846155e-06,
"loss": 0.9105,
"step": 4
},
{
"epoch": 0.01,
"learning_rate": 7.692307692307694e-06,
"loss": 0.9502,
"step": 5
},
{
"epoch": 0.01,
"learning_rate": 9.230769230769232e-06,
"loss": 0.9663,
"step": 6
},
{
"epoch": 0.02,
"learning_rate": 1.076923076923077e-05,
"loss": 0.9044,
"step": 7
},
{
"epoch": 0.02,
"learning_rate": 1.230769230769231e-05,
"loss": 0.9281,
"step": 8
},
{
"epoch": 0.02,
"learning_rate": 1.3846153846153847e-05,
"loss": 0.8729,
"step": 9
},
{
"epoch": 0.02,
"learning_rate": 1.5384615384615387e-05,
"loss": 0.894,
"step": 10
},
{
"epoch": 0.03,
"learning_rate": 1.6923076923076924e-05,
"loss": 0.9235,
"step": 11
},
{
"epoch": 0.03,
"learning_rate": 1.8461538461538465e-05,
"loss": 0.905,
"step": 12
},
{
"epoch": 0.03,
"learning_rate": 2e-05,
"loss": 0.908,
"step": 13
},
{
"epoch": 0.03,
"learning_rate": 1.9999710687264073e-05,
"loss": 0.9524,
"step": 14
},
{
"epoch": 0.04,
"learning_rate": 1.999884276579666e-05,
"loss": 0.9495,
"step": 15
},
{
"epoch": 0.04,
"learning_rate": 1.9997396285817907e-05,
"loss": 0.8929,
"step": 16
},
{
"epoch": 0.04,
"learning_rate": 1.9995371331024835e-05,
"loss": 0.9357,
"step": 17
},
{
"epoch": 0.04,
"learning_rate": 1.999276801858648e-05,
"loss": 0.9182,
"step": 18
},
{
"epoch": 0.04,
"learning_rate": 1.9989586499137137e-05,
"loss": 0.9526,
"step": 19
},
{
"epoch": 0.05,
"learning_rate": 1.998582695676762e-05,
"loss": 0.9119,
"step": 20
},
{
"epoch": 0.05,
"learning_rate": 1.998148960901463e-05,
"loss": 0.9072,
"step": 21
},
{
"epoch": 0.05,
"learning_rate": 1.9976574706848154e-05,
"loss": 0.888,
"step": 22
},
{
"epoch": 0.05,
"learning_rate": 1.9971082534656958e-05,
"loss": 0.9345,
"step": 23
},
{
"epoch": 0.06,
"learning_rate": 1.99650134102321e-05,
"loss": 0.954,
"step": 24
},
{
"epoch": 0.06,
"learning_rate": 1.995836768474859e-05,
"loss": 0.9353,
"step": 25
},
{
"epoch": 0.06,
"learning_rate": 1.9951145742745027e-05,
"loss": 0.8974,
"step": 26
},
{
"epoch": 0.06,
"learning_rate": 1.9943348002101374e-05,
"loss": 0.9084,
"step": 27
},
{
"epoch": 0.07,
"learning_rate": 1.9934974914014765e-05,
"loss": 0.9109,
"step": 28
},
{
"epoch": 0.07,
"learning_rate": 1.9926026962973407e-05,
"loss": 0.9312,
"step": 29
},
{
"epoch": 0.07,
"learning_rate": 1.9916504666728533e-05,
"loss": 0.8948,
"step": 30
},
{
"epoch": 0.07,
"learning_rate": 1.9906408576264468e-05,
"loss": 0.8773,
"step": 31
},
{
"epoch": 0.08,
"learning_rate": 1.9895739275766717e-05,
"loss": 0.9831,
"step": 32
},
{
"epoch": 0.08,
"learning_rate": 1.9884497382588185e-05,
"loss": 0.9739,
"step": 33
},
{
"epoch": 0.08,
"learning_rate": 1.9872683547213446e-05,
"loss": 0.9398,
"step": 34
},
{
"epoch": 0.08,
"learning_rate": 1.986029845322111e-05,
"loss": 0.8877,
"step": 35
},
{
"epoch": 0.08,
"learning_rate": 1.9847342817244256e-05,
"loss": 0.8749,
"step": 36
},
{
"epoch": 0.09,
"learning_rate": 1.9833817388928987e-05,
"loss": 0.9456,
"step": 37
},
{
"epoch": 0.09,
"learning_rate": 1.9819722950891034e-05,
"loss": 0.9641,
"step": 38
},
{
"epoch": 0.09,
"learning_rate": 1.980506031867049e-05,
"loss": 0.8627,
"step": 39
},
{
"epoch": 0.09,
"learning_rate": 1.9789830340684593e-05,
"loss": 0.941,
"step": 40
},
{
"epoch": 0.1,
"learning_rate": 1.9774033898178668e-05,
"loss": 0.9602,
"step": 41
},
{
"epoch": 0.1,
"learning_rate": 1.9757671905175118e-05,
"loss": 0.9361,
"step": 42
},
{
"epoch": 0.1,
"learning_rate": 1.974074530842053e-05,
"loss": 0.8819,
"step": 43
},
{
"epoch": 0.1,
"learning_rate": 1.972325508733091e-05,
"loss": 0.9098,
"step": 44
},
{
"epoch": 0.11,
"learning_rate": 1.9705202253935004e-05,
"loss": 0.974,
"step": 45
},
{
"epoch": 0.11,
"learning_rate": 1.9686587852815733e-05,
"loss": 0.9917,
"step": 46
},
{
"epoch": 0.11,
"learning_rate": 1.9667412961049755e-05,
"loss": 0.9431,
"step": 47
},
{
"epoch": 0.11,
"learning_rate": 1.9647678688145163e-05,
"loss": 0.8762,
"step": 48
},
{
"epoch": 0.12,
"learning_rate": 1.962738617597724e-05,
"loss": 0.8853,
"step": 49
},
{
"epoch": 0.12,
"learning_rate": 1.9606536598722435e-05,
"loss": 0.8557,
"step": 50
},
{
"epoch": 0.12,
"learning_rate": 1.95851311627904e-05,
"loss": 0.8813,
"step": 51
},
{
"epoch": 0.12,
"learning_rate": 1.956317110675417e-05,
"loss": 0.9625,
"step": 52
},
{
"epoch": 0.12,
"learning_rate": 1.9540657701278536e-05,
"loss": 0.9091,
"step": 53
},
{
"epoch": 0.13,
"learning_rate": 1.9517592249046476e-05,
"loss": 0.9151,
"step": 54
},
{
"epoch": 0.13,
"learning_rate": 1.9493976084683814e-05,
"loss": 0.9429,
"step": 55
},
{
"epoch": 0.13,
"learning_rate": 1.946981057468197e-05,
"loss": 0.8978,
"step": 56
},
{
"epoch": 0.13,
"learning_rate": 1.9445097117318915e-05,
"loss": 0.9288,
"step": 57
},
{
"epoch": 0.14,
"learning_rate": 1.9419837142578228e-05,
"loss": 0.9247,
"step": 58
},
{
"epoch": 0.14,
"learning_rate": 1.9394032112066405e-05,
"loss": 0.8868,
"step": 59
},
{
"epoch": 0.14,
"learning_rate": 1.9367683518928228e-05,
"loss": 0.9086,
"step": 60
},
{
"epoch": 0.14,
"learning_rate": 1.9340792887760417e-05,
"loss": 0.8921,
"step": 61
},
{
"epoch": 0.15,
"learning_rate": 1.9313361774523387e-05,
"loss": 0.8934,
"step": 62
},
{
"epoch": 0.15,
"learning_rate": 1.928539176645122e-05,
"loss": 0.8898,
"step": 63
},
{
"epoch": 0.15,
"learning_rate": 1.925688448195983e-05,
"loss": 0.9609,
"step": 64
},
{
"epoch": 0.15,
"learning_rate": 1.922784157055331e-05,
"loss": 0.931,
"step": 65
},
{
"epoch": 0.15,
"learning_rate": 1.9198264712728494e-05,
"loss": 0.9022,
"step": 66
},
{
"epoch": 0.16,
"learning_rate": 1.9168155619877712e-05,
"loss": 0.8409,
"step": 67
},
{
"epoch": 0.16,
"learning_rate": 1.9137516034189768e-05,
"loss": 0.9554,
"step": 68
},
{
"epoch": 0.16,
"learning_rate": 1.9106347728549134e-05,
"loss": 0.8817,
"step": 69
},
{
"epoch": 0.16,
"learning_rate": 1.907465250643337e-05,
"loss": 0.9268,
"step": 70
},
{
"epoch": 0.17,
"learning_rate": 1.9042432201808757e-05,
"loss": 0.9427,
"step": 71
},
{
"epoch": 0.17,
"learning_rate": 1.900968867902419e-05,
"loss": 0.9137,
"step": 72
},
{
"epoch": 0.17,
"learning_rate": 1.897642383270331e-05,
"loss": 0.8941,
"step": 73
},
{
"epoch": 0.17,
"learning_rate": 1.8942639587634854e-05,
"loss": 0.8784,
"step": 74
},
{
"epoch": 0.18,
"learning_rate": 1.8908337898661287e-05,
"loss": 0.9041,
"step": 75
},
{
"epoch": 0.18,
"learning_rate": 1.8873520750565716e-05,
"loss": 0.8818,
"step": 76
},
{
"epoch": 0.18,
"learning_rate": 1.8838190157957013e-05,
"loss": 0.8648,
"step": 77
},
{
"epoch": 0.18,
"learning_rate": 1.880234816515326e-05,
"loss": 0.9294,
"step": 78
},
{
"epoch": 0.19,
"learning_rate": 1.8765996846063456e-05,
"loss": 0.9423,
"step": 79
},
{
"epoch": 0.19,
"learning_rate": 1.8729138304067512e-05,
"loss": 0.9467,
"step": 80
},
{
"epoch": 0.19,
"learning_rate": 1.8691774671894562e-05,
"loss": 0.8737,
"step": 81
},
{
"epoch": 0.19,
"learning_rate": 1.8653908111499534e-05,
"loss": 0.8909,
"step": 82
},
{
"epoch": 0.19,
"learning_rate": 1.8615540813938063e-05,
"loss": 0.912,
"step": 83
},
{
"epoch": 0.2,
"learning_rate": 1.8576674999239713e-05,
"loss": 0.9153,
"step": 84
},
{
"epoch": 0.2,
"learning_rate": 1.8537312916279526e-05,
"loss": 0.9053,
"step": 85
},
{
"epoch": 0.2,
"learning_rate": 1.849745684264788e-05,
"loss": 0.9688,
"step": 86
},
{
"epoch": 0.2,
"learning_rate": 1.845710908451872e-05,
"loss": 0.9658,
"step": 87
},
{
"epoch": 0.21,
"learning_rate": 1.84162719765161e-05,
"loss": 0.9626,
"step": 88
},
{
"epoch": 0.21,
"learning_rate": 1.8374947881579116e-05,
"loss": 0.9328,
"step": 89
},
{
"epoch": 0.21,
"learning_rate": 1.833313919082515e-05,
"loss": 0.9245,
"step": 90
},
{
"epoch": 0.21,
"learning_rate": 1.8290848323411554e-05,
"loss": 0.8959,
"step": 91
},
{
"epoch": 0.22,
"learning_rate": 1.8248077726395634e-05,
"loss": 0.8694,
"step": 92
},
{
"epoch": 0.22,
"learning_rate": 1.8204829874593083e-05,
"loss": 0.8948,
"step": 93
},
{
"epoch": 0.22,
"learning_rate": 1.8161107270434757e-05,
"loss": 0.9032,
"step": 94
},
{
"epoch": 0.22,
"learning_rate": 1.8116912443821912e-05,
"loss": 0.8311,
"step": 95
},
{
"epoch": 0.23,
"learning_rate": 1.8072247951979785e-05,
"loss": 0.9178,
"step": 96
},
{
"epoch": 0.23,
"learning_rate": 1.8027116379309637e-05,
"loss": 0.9193,
"step": 97
},
{
"epoch": 0.23,
"learning_rate": 1.798152033723923e-05,
"loss": 0.8684,
"step": 98
},
{
"epoch": 0.23,
"learning_rate": 1.7935462464071697e-05,
"loss": 0.8673,
"step": 99
},
{
"epoch": 0.23,
"learning_rate": 1.7888945424832896e-05,
"loss": 0.8611,
"step": 100
},
{
"epoch": 0.24,
"learning_rate": 1.78419719111172e-05,
"loss": 0.9236,
"step": 101
},
{
"epoch": 0.24,
"learning_rate": 1.7794544640931773e-05,
"loss": 0.8275,
"step": 102
},
{
"epoch": 0.24,
"learning_rate": 1.774666635853927e-05,
"loss": 0.867,
"step": 103
},
{
"epoch": 0.24,
"learning_rate": 1.7698339834299064e-05,
"loss": 0.9036,
"step": 104
},
{
"epoch": 0.25,
"learning_rate": 1.7649567864506943e-05,
"loss": 0.9314,
"step": 105
},
{
"epoch": 0.25,
"learning_rate": 1.7600353271233316e-05,
"loss": 0.8757,
"step": 106
},
{
"epoch": 0.25,
"learning_rate": 1.7550698902159897e-05,
"loss": 0.9196,
"step": 107
},
{
"epoch": 0.25,
"learning_rate": 1.7500607630414973e-05,
"loss": 0.8508,
"step": 108
},
{
"epoch": 0.26,
"learning_rate": 1.745008235440711e-05,
"loss": 0.8699,
"step": 109
},
{
"epoch": 0.26,
"learning_rate": 1.7399125997657478e-05,
"loss": 0.8734,
"step": 110
},
{
"epoch": 0.26,
"learning_rate": 1.7347741508630673e-05,
"loss": 0.8488,
"step": 111
},
{
"epoch": 0.26,
"learning_rate": 1.729593186056412e-05,
"loss": 0.8975,
"step": 112
},
{
"epoch": 0.27,
"learning_rate": 1.7243700051296016e-05,
"loss": 0.9325,
"step": 113
},
{
"epoch": 0.27,
"learning_rate": 1.71910491030919e-05,
"loss": 0.8745,
"step": 114
},
{
"epoch": 0.27,
"learning_rate": 1.713798206246974e-05,
"loss": 0.945,
"step": 115
},
{
"epoch": 0.27,
"learning_rate": 1.708450200002368e-05,
"loss": 0.9282,
"step": 116
},
{
"epoch": 0.27,
"learning_rate": 1.703061201024636e-05,
"loss": 0.8688,
"step": 117
},
{
"epoch": 0.28,
"learning_rate": 1.6976315211349848e-05,
"loss": 0.8962,
"step": 118
},
{
"epoch": 0.28,
"learning_rate": 1.6921614745085236e-05,
"loss": 0.8591,
"step": 119
},
{
"epoch": 0.28,
"learning_rate": 1.6866513776560844e-05,
"loss": 0.8918,
"step": 120
},
{
"epoch": 0.28,
"learning_rate": 1.6811015494059048e-05,
"loss": 0.9125,
"step": 121
},
{
"epoch": 0.29,
"learning_rate": 1.6755123108851846e-05,
"loss": 0.8773,
"step": 122
},
{
"epoch": 0.29,
"learning_rate": 1.669883985501501e-05,
"loss": 0.9334,
"step": 123
},
{
"epoch": 0.29,
"learning_rate": 1.6642168989240977e-05,
"loss": 0.887,
"step": 124
},
{
"epoch": 0.29,
"learning_rate": 1.658511379065039e-05,
"loss": 0.8577,
"step": 125
},
{
"epoch": 0.3,
"learning_rate": 1.6527677560602364e-05,
"loss": 0.8751,
"step": 126
},
{
"epoch": 0.3,
"learning_rate": 1.6469863622503478e-05,
"loss": 0.8974,
"step": 127
},
{
"epoch": 0.3,
"learning_rate": 1.641167532161545e-05,
"loss": 0.9063,
"step": 128
},
{
"epoch": 0.3,
"learning_rate": 1.6353116024861584e-05,
"loss": 0.8761,
"step": 129
},
{
"epoch": 0.31,
"learning_rate": 1.6294189120631956e-05,
"loss": 0.8967,
"step": 130
},
{
"epoch": 0.31,
"learning_rate": 1.6234898018587336e-05,
"loss": 0.8966,
"step": 131
},
{
"epoch": 0.31,
"learning_rate": 1.617524614946192e-05,
"loss": 0.901,
"step": 132
},
{
"epoch": 0.31,
"learning_rate": 1.6115236964864798e-05,
"loss": 0.843,
"step": 133
},
{
"epoch": 0.31,
"learning_rate": 1.6054873937080248e-05,
"loss": 0.8904,
"step": 134
},
{
"epoch": 0.32,
"learning_rate": 1.5994160558866803e-05,
"loss": 0.9368,
"step": 135
},
{
"epoch": 0.32,
"learning_rate": 1.5933100343255184e-05,
"loss": 0.9121,
"step": 136
},
{
"epoch": 0.32,
"learning_rate": 1.5871696823344998e-05,
"loss": 0.928,
"step": 137
},
{
"epoch": 0.32,
"learning_rate": 1.580995355210031e-05,
"loss": 0.8595,
"step": 138
},
{
"epoch": 0.33,
"learning_rate": 1.5747874102144073e-05,
"loss": 0.827,
"step": 139
},
{
"epoch": 0.33,
"learning_rate": 1.5685462065551375e-05,
"loss": 0.8894,
"step": 140
},
{
"epoch": 0.33,
"learning_rate": 1.562272105364164e-05,
"loss": 0.8695,
"step": 141
},
{
"epoch": 0.33,
"learning_rate": 1.5559654696769628e-05,
"loss": 0.8895,
"step": 142
},
{
"epoch": 0.34,
"learning_rate": 1.5496266644115386e-05,
"loss": 0.9051,
"step": 143
},
{
"epoch": 0.34,
"learning_rate": 1.5432560563473106e-05,
"loss": 0.8599,
"step": 144
},
{
"epoch": 0.34,
"learning_rate": 1.536854014103888e-05,
"loss": 0.8624,
"step": 145
},
{
"epoch": 0.34,
"learning_rate": 1.5304209081197425e-05,
"loss": 0.8795,
"step": 146
},
{
"epoch": 0.35,
"learning_rate": 1.5239571106307729e-05,
"loss": 0.8165,
"step": 147
},
{
"epoch": 0.35,
"learning_rate": 1.5174629956487659e-05,
"loss": 0.8826,
"step": 148
},
{
"epoch": 0.35,
"learning_rate": 1.5109389389397566e-05,
"loss": 0.8126,
"step": 149
},
{
"epoch": 0.35,
"learning_rate": 1.5043853180022838e-05,
"loss": 0.8531,
"step": 150
},
{
"epoch": 0.35,
"learning_rate": 1.4978025120455484e-05,
"loss": 0.8569,
"step": 151
},
{
"epoch": 0.36,
"learning_rate": 1.4911909019674703e-05,
"loss": 0.8385,
"step": 152
},
{
"epoch": 0.36,
"learning_rate": 1.4845508703326504e-05,
"loss": 0.8987,
"step": 153
},
{
"epoch": 0.36,
"learning_rate": 1.4778828013502318e-05,
"loss": 0.8636,
"step": 154
},
{
"epoch": 0.36,
"learning_rate": 1.4711870808516708e-05,
"loss": 0.8986,
"step": 155
},
{
"epoch": 0.37,
"learning_rate": 1.4644640962684107e-05,
"loss": 0.8355,
"step": 156
},
{
"epoch": 0.37,
"learning_rate": 1.4577142366094643e-05,
"loss": 0.8899,
"step": 157
},
{
"epoch": 0.37,
"learning_rate": 1.4509378924389044e-05,
"loss": 0.8985,
"step": 158
},
{
"epoch": 0.37,
"learning_rate": 1.4441354558532654e-05,
"loss": 0.8431,
"step": 159
},
{
"epoch": 0.38,
"learning_rate": 1.4373073204588556e-05,
"loss": 0.8805,
"step": 160
},
{
"epoch": 0.38,
"learning_rate": 1.4304538813489808e-05,
"loss": 0.8278,
"step": 161
},
{
"epoch": 0.38,
"learning_rate": 1.4235755350810854e-05,
"loss": 0.8554,
"step": 162
},
{
"epoch": 0.38,
"learning_rate": 1.4166726796538044e-05,
"loss": 0.8091,
"step": 163
},
{
"epoch": 0.38,
"learning_rate": 1.4097457144839362e-05,
"loss": 0.8755,
"step": 164
},
{
"epoch": 0.39,
"learning_rate": 1.4027950403833295e-05,
"loss": 0.8718,
"step": 165
},
{
"epoch": 0.39,
"learning_rate": 1.3958210595356924e-05,
"loss": 0.8415,
"step": 166
},
{
"epoch": 0.39,
"learning_rate": 1.388824175473321e-05,
"loss": 0.8597,
"step": 167
},
{
"epoch": 0.39,
"learning_rate": 1.3818047930537491e-05,
"loss": 0.9381,
"step": 168
},
{
"epoch": 0.4,
"learning_rate": 1.3747633184363234e-05,
"loss": 0.8926,
"step": 169
},
{
"epoch": 0.4,
"learning_rate": 1.3677001590587011e-05,
"loss": 0.8472,
"step": 170
},
{
"epoch": 0.4,
"learning_rate": 1.3606157236132756e-05,
"loss": 0.8711,
"step": 171
},
{
"epoch": 0.4,
"learning_rate": 1.3535104220235262e-05,
"loss": 0.8187,
"step": 172
},
{
"epoch": 0.41,
"learning_rate": 1.3463846654203021e-05,
"loss": 0.8371,
"step": 173
},
{
"epoch": 0.41,
"learning_rate": 1.3392388661180303e-05,
"loss": 0.8185,
"step": 174
},
{
"epoch": 0.41,
"learning_rate": 1.3320734375908608e-05,
"loss": 0.8315,
"step": 175
},
{
"epoch": 0.41,
"learning_rate": 1.3248887944487397e-05,
"loss": 0.8923,
"step": 176
},
{
"epoch": 0.42,
"learning_rate": 1.3176853524134198e-05,
"loss": 0.8946,
"step": 177
},
{
"epoch": 0.42,
"learning_rate": 1.3104635282944054e-05,
"loss": 0.8792,
"step": 178
},
{
"epoch": 0.42,
"learning_rate": 1.3032237399648357e-05,
"loss": 0.8519,
"step": 179
},
{
"epoch": 0.42,
"learning_rate": 1.2959664063373044e-05,
"loss": 0.8592,
"step": 180
},
{
"epoch": 0.42,
"learning_rate": 1.2886919473396212e-05,
"loss": 0.8558,
"step": 181
},
{
"epoch": 0.43,
"learning_rate": 1.281400783890513e-05,
"loss": 0.8826,
"step": 182
},
{
"epoch": 0.43,
"learning_rate": 1.2740933378752685e-05,
"loss": 0.8083,
"step": 183
},
{
"epoch": 0.43,
"learning_rate": 1.2667700321213282e-05,
"loss": 0.863,
"step": 184
},
{
"epoch": 0.43,
"learning_rate": 1.2594312903738164e-05,
"loss": 0.8179,
"step": 185
},
{
"epoch": 0.44,
"learning_rate": 1.2520775372710242e-05,
"loss": 0.785,
"step": 186
},
{
"epoch": 0.44,
"learning_rate": 1.2447091983198368e-05,
"loss": 0.8361,
"step": 187
},
{
"epoch": 0.44,
"learning_rate": 1.2373266998711152e-05,
"loss": 0.8412,
"step": 188
},
{
"epoch": 0.44,
"learning_rate": 1.2299304690950235e-05,
"loss": 0.8854,
"step": 189
},
{
"epoch": 0.45,
"learning_rate": 1.2225209339563144e-05,
"loss": 0.8221,
"step": 190
},
{
"epoch": 0.45,
"learning_rate": 1.2150985231895647e-05,
"loss": 0.8823,
"step": 191
},
{
"epoch": 0.45,
"learning_rate": 1.2076636662743673e-05,
"loss": 0.905,
"step": 192
},
{
"epoch": 0.45,
"learning_rate": 1.2002167934104815e-05,
"loss": 0.9039,
"step": 193
},
{
"epoch": 0.46,
"learning_rate": 1.1927583354929393e-05,
"loss": 0.8522,
"step": 194
},
{
"epoch": 0.46,
"learning_rate": 1.1852887240871145e-05,
"loss": 0.8775,
"step": 195
},
{
"epoch": 0.46,
"learning_rate": 1.177808391403749e-05,
"loss": 0.8461,
"step": 196
},
{
"epoch": 0.46,
"learning_rate": 1.170317770273946e-05,
"loss": 0.8631,
"step": 197
},
{
"epoch": 0.46,
"learning_rate": 1.162817294124124e-05,
"loss": 0.8432,
"step": 198
},
{
"epoch": 0.47,
"learning_rate": 1.1553073969509382e-05,
"loss": 0.8647,
"step": 199
},
{
"epoch": 0.47,
"learning_rate": 1.1477885132961679e-05,
"loss": 0.815,
"step": 200
},
{
"epoch": 0.47,
"learning_rate": 1.1402610782215735e-05,
"loss": 0.8566,
"step": 201
},
{
"epoch": 0.47,
"learning_rate": 1.1327255272837221e-05,
"loss": 0.8163,
"step": 202
},
{
"epoch": 0.48,
"learning_rate": 1.1251822965087856e-05,
"loss": 0.8177,
"step": 203
},
{
"epoch": 0.48,
"learning_rate": 1.1176318223673106e-05,
"loss": 0.9081,
"step": 204
},
{
"epoch": 0.48,
"learning_rate": 1.1100745417489631e-05,
"loss": 0.8451,
"step": 205
},
{
"epoch": 0.48,
"learning_rate": 1.1025108919372501e-05,
"loss": 0.8246,
"step": 206
},
{
"epoch": 0.49,
"learning_rate": 1.0949413105842148e-05,
"loss": 0.8439,
"step": 207
},
{
"epoch": 0.49,
"learning_rate": 1.0873662356851164e-05,
"loss": 0.8444,
"step": 208
},
{
"epoch": 0.49,
"learning_rate": 1.0797861055530832e-05,
"loss": 0.852,
"step": 209
},
{
"epoch": 0.49,
"learning_rate": 1.0722013587937528e-05,
"loss": 0.8598,
"step": 210
},
{
"epoch": 0.5,
"learning_rate": 1.064612434279892e-05,
"loss": 0.896,
"step": 211
},
{
"epoch": 0.5,
"learning_rate": 1.057019771126004e-05,
"loss": 0.7676,
"step": 212
},
{
"epoch": 0.5,
"learning_rate": 1.0494238086629184e-05,
"loss": 0.8303,
"step": 213
},
{
"epoch": 0.5,
"learning_rate": 1.0418249864123724e-05,
"loss": 0.8253,
"step": 214
},
{
"epoch": 0.5,
"learning_rate": 1.0342237440615765e-05,
"loss": 0.8391,
"step": 215
},
{
"epoch": 0.51,
"learning_rate": 1.026620521437775e-05,
"loss": 0.8656,
"step": 216
},
{
"epoch": 0.51,
"learning_rate": 1.0190157584827952e-05,
"loss": 0.8424,
"step": 217
},
{
"epoch": 0.51,
"learning_rate": 1.0114098952275935e-05,
"loss": 0.8154,
"step": 218
},
{
"epoch": 0.51,
"learning_rate": 1.0038033717667907e-05,
"loss": 0.8372,
"step": 219
},
{
"epoch": 0.52,
"learning_rate": 9.961966282332094e-06,
"loss": 0.8529,
"step": 220
},
{
"epoch": 0.52,
"learning_rate": 9.885901047724066e-06,
"loss": 0.8619,
"step": 221
},
{
"epoch": 0.52,
"learning_rate": 9.80984241517205e-06,
"loss": 0.7912,
"step": 222
},
{
"epoch": 0.52,
"learning_rate": 9.733794785622254e-06,
"loss": 0.8761,
"step": 223
},
{
"epoch": 0.53,
"learning_rate": 9.657762559384237e-06,
"loss": 0.8112,
"step": 224
},
{
"epoch": 0.53,
"learning_rate": 9.581750135876277e-06,
"loss": 0.8028,
"step": 225
},
{
"epoch": 0.53,
"learning_rate": 9.505761913370814e-06,
"loss": 0.8751,
"step": 226
},
{
"epoch": 0.53,
"learning_rate": 9.429802288739963e-06,
"loss": 0.8743,
"step": 227
},
{
"epoch": 0.54,
"learning_rate": 9.353875657201084e-06,
"loss": 0.8358,
"step": 228
},
{
"epoch": 0.54,
"learning_rate": 9.277986412062476e-06,
"loss": 0.7817,
"step": 229
},
{
"epoch": 0.54,
"learning_rate": 9.202138944469168e-06,
"loss": 0.8076,
"step": 230
},
{
"epoch": 0.54,
"learning_rate": 9.12633764314884e-06,
"loss": 0.8662,
"step": 231
},
{
"epoch": 0.54,
"learning_rate": 9.050586894157853e-06,
"loss": 0.8079,
"step": 232
},
{
"epoch": 0.55,
"learning_rate": 8.974891080627504e-06,
"loss": 0.8166,
"step": 233
},
{
"epoch": 0.55,
"learning_rate": 8.899254582510369e-06,
"loss": 0.8975,
"step": 234
},
{
"epoch": 0.55,
"learning_rate": 8.823681776326899e-06,
"loss": 0.7989,
"step": 235
},
{
"epoch": 0.55,
"learning_rate": 8.748177034912148e-06,
"loss": 0.8004,
"step": 236
},
{
"epoch": 0.56,
"learning_rate": 8.672744727162782e-06,
"loss": 0.8065,
"step": 237
},
{
"epoch": 0.56,
"learning_rate": 8.597389217784268e-06,
"loss": 0.799,
"step": 238
},
{
"epoch": 0.56,
"learning_rate": 8.522114867038323e-06,
"loss": 0.8044,
"step": 239
},
{
"epoch": 0.56,
"learning_rate": 8.446926030490623e-06,
"loss": 0.7698,
"step": 240
},
{
"epoch": 0.57,
"learning_rate": 8.371827058758763e-06,
"loss": 0.8325,
"step": 241
},
{
"epoch": 0.57,
"learning_rate": 8.296822297260541e-06,
"loss": 0.8511,
"step": 242
},
{
"epoch": 0.57,
"learning_rate": 8.221916085962511e-06,
"loss": 0.8323,
"step": 243
},
{
"epoch": 0.57,
"learning_rate": 8.147112759128859e-06,
"loss": 0.8127,
"step": 244
},
{
"epoch": 0.58,
"learning_rate": 8.072416645070609e-06,
"loss": 0.8726,
"step": 245
},
{
"epoch": 0.58,
"learning_rate": 7.997832065895188e-06,
"loss": 0.824,
"step": 246
},
{
"epoch": 0.58,
"learning_rate": 7.923363337256327e-06,
"loss": 0.7932,
"step": 247
},
{
"epoch": 0.58,
"learning_rate": 7.849014768104354e-06,
"loss": 0.8338,
"step": 248
},
{
"epoch": 0.58,
"learning_rate": 7.774790660436857e-06,
"loss": 0.8015,
"step": 249
},
{
"epoch": 0.59,
"learning_rate": 7.700695309049768e-06,
"loss": 0.7834,
"step": 250
},
{
"epoch": 0.59,
"learning_rate": 7.626733001288852e-06,
"loss": 0.8021,
"step": 251
},
{
"epoch": 0.59,
"learning_rate": 7.552908016801633e-06,
"loss": 0.8361,
"step": 252
},
{
"epoch": 0.59,
"learning_rate": 7.479224627289765e-06,
"loss": 0.8649,
"step": 253
},
{
"epoch": 0.6,
"learning_rate": 7.40568709626184e-06,
"loss": 0.8319,
"step": 254
},
{
"epoch": 0.6,
"learning_rate": 7.3322996787867225e-06,
"loss": 0.8105,
"step": 255
},
{
"epoch": 0.6,
"learning_rate": 7.259066621247316e-06,
"loss": 0.878,
"step": 256
},
{
"epoch": 0.6,
"learning_rate": 7.185992161094877e-06,
"loss": 0.7354,
"step": 257
},
{
"epoch": 0.61,
"learning_rate": 7.113080526603793e-06,
"loss": 0.8544,
"step": 258
},
{
"epoch": 0.61,
"learning_rate": 7.040335936626959e-06,
"loss": 0.8877,
"step": 259
},
{
"epoch": 0.61,
"learning_rate": 6.967762600351646e-06,
"loss": 0.8115,
"step": 260
},
{
"epoch": 0.61,
"learning_rate": 6.895364717055949e-06,
"loss": 0.8131,
"step": 261
},
{
"epoch": 0.62,
"learning_rate": 6.8231464758658075e-06,
"loss": 0.8128,
"step": 262
},
{
"epoch": 0.62,
"learning_rate": 6.7511120555126055e-06,
"loss": 0.8055,
"step": 263
},
{
"epoch": 0.62,
"learning_rate": 6.6792656240913936e-06,
"loss": 0.8022,
"step": 264
},
{
"epoch": 0.62,
"learning_rate": 6.607611338819697e-06,
"loss": 0.8304,
"step": 265
},
{
"epoch": 0.62,
"learning_rate": 6.536153345796984e-06,
"loss": 0.8399,
"step": 266
},
{
"epoch": 0.63,
"learning_rate": 6.464895779764742e-06,
"loss": 0.7836,
"step": 267
},
{
"epoch": 0.63,
"learning_rate": 6.393842763867248e-06,
"loss": 0.7966,
"step": 268
},
{
"epoch": 0.63,
"learning_rate": 6.3229984094129895e-06,
"loss": 0.8371,
"step": 269
},
{
"epoch": 0.63,
"learning_rate": 6.252366815636768e-06,
"loss": 0.8918,
"step": 270
},
{
"epoch": 0.64,
"learning_rate": 6.181952069462514e-06,
"loss": 0.819,
"step": 271
},
{
"epoch": 0.64,
"learning_rate": 6.111758245266795e-06,
"loss": 0.824,
"step": 272
},
{
"epoch": 0.64,
"learning_rate": 6.041789404643078e-06,
"loss": 0.8308,
"step": 273
},
{
"epoch": 0.64,
"learning_rate": 5.972049596166707e-06,
"loss": 0.8239,
"step": 274
},
{
"epoch": 0.65,
"learning_rate": 5.902542855160642e-06,
"loss": 0.798,
"step": 275
},
{
"epoch": 0.65,
"learning_rate": 5.83327320346196e-06,
"loss": 0.8063,
"step": 276
},
{
"epoch": 0.65,
"learning_rate": 5.76424464918915e-06,
"loss": 0.7963,
"step": 277
},
{
"epoch": 0.65,
"learning_rate": 5.695461186510194e-06,
"loss": 0.8204,
"step": 278
},
{
"epoch": 0.65,
"learning_rate": 5.626926795411447e-06,
"loss": 0.8204,
"step": 279
},
{
"epoch": 0.66,
"learning_rate": 5.558645441467347e-06,
"loss": 0.7832,
"step": 280
},
{
"epoch": 0.66,
"learning_rate": 5.490621075610958e-06,
"loss": 0.8548,
"step": 281
},
{
"epoch": 0.66,
"learning_rate": 5.422857633905358e-06,
"loss": 0.8022,
"step": 282
},
{
"epoch": 0.66,
"learning_rate": 5.355359037315893e-06,
"loss": 0.8975,
"step": 283
},
{
"epoch": 0.67,
"learning_rate": 5.288129191483297e-06,
"loss": 0.8389,
"step": 284
},
{
"epoch": 0.67,
"learning_rate": 5.221171986497686e-06,
"loss": 0.8377,
"step": 285
},
{
"epoch": 0.67,
"learning_rate": 5.1544912966735e-06,
"loss": 0.7948,
"step": 286
},
{
"epoch": 0.67,
"learning_rate": 5.0880909803252974e-06,
"loss": 0.7896,
"step": 287
},
{
"epoch": 0.68,
"learning_rate": 5.021974879544522e-06,
"loss": 0.8582,
"step": 288
},
{
"epoch": 0.68,
"learning_rate": 4.956146819977166e-06,
"loss": 0.842,
"step": 289
},
{
"epoch": 0.68,
"learning_rate": 4.890610610602437e-06,
"loss": 0.8021,
"step": 290
},
{
"epoch": 0.68,
"learning_rate": 4.82537004351234e-06,
"loss": 0.7894,
"step": 291
},
{
"epoch": 0.69,
"learning_rate": 4.760428893692274e-06,
"loss": 0.8664,
"step": 292
},
{
"epoch": 0.69,
"learning_rate": 4.695790918802577e-06,
"loss": 0.8016,
"step": 293
},
{
"epoch": 0.69,
"learning_rate": 4.631459858961123e-06,
"loss": 0.8085,
"step": 294
},
{
"epoch": 0.69,
"learning_rate": 4.567439436526897e-06,
"loss": 0.8079,
"step": 295
},
{
"epoch": 0.69,
"learning_rate": 4.5037333558846155e-06,
"loss": 0.782,
"step": 296
},
{
"epoch": 0.7,
"learning_rate": 4.440345303230377e-06,
"loss": 0.7886,
"step": 297
},
{
"epoch": 0.7,
"learning_rate": 4.377278946358363e-06,
"loss": 0.8899,
"step": 298
},
{
"epoch": 0.7,
"learning_rate": 4.314537934448628e-06,
"loss": 0.8182,
"step": 299
},
{
"epoch": 0.7,
"learning_rate": 4.2521258978559324e-06,
"loss": 0.8336,
"step": 300
},
{
"epoch": 0.71,
"learning_rate": 4.19004644789969e-06,
"loss": 0.8754,
"step": 301
},
{
"epoch": 0.71,
"learning_rate": 4.128303176655002e-06,
"loss": 0.7593,
"step": 302
},
{
"epoch": 0.71,
"learning_rate": 4.066899656744816e-06,
"loss": 0.8286,
"step": 303
},
{
"epoch": 0.71,
"learning_rate": 4.005839441133198e-06,
"loss": 0.8598,
"step": 304
},
{
"epoch": 0.72,
"learning_rate": 3.945126062919756e-06,
"loss": 0.8381,
"step": 305
},
{
"epoch": 0.72,
"learning_rate": 3.884763035135205e-06,
"loss": 0.7859,
"step": 306
},
{
"epoch": 0.72,
"learning_rate": 3.824753850538082e-06,
"loss": 0.8469,
"step": 307
},
{
"epoch": 0.72,
"learning_rate": 3.7651019814126656e-06,
"loss": 0.8343,
"step": 308
},
{
"epoch": 0.73,
"learning_rate": 3.7058108793680467e-06,
"loss": 0.8597,
"step": 309
},
{
"epoch": 0.73,
"learning_rate": 3.646883975138421e-06,
"loss": 0.8567,
"step": 310
},
{
"epoch": 0.73,
"learning_rate": 3.588324678384555e-06,
"loss": 0.7813,
"step": 311
},
{
"epoch": 0.73,
"learning_rate": 3.5301363774965256e-06,
"loss": 0.7459,
"step": 312
},
{
"epoch": 0.73,
"learning_rate": 3.4723224393976353e-06,
"loss": 0.8251,
"step": 313
},
{
"epoch": 0.74,
"learning_rate": 3.414886209349615e-06,
"loss": 0.8412,
"step": 314
},
{
"epoch": 0.74,
"learning_rate": 3.357831010759026e-06,
"loss": 0.7613,
"step": 315
},
{
"epoch": 0.74,
"learning_rate": 3.301160144984992e-06,
"loss": 0.8462,
"step": 316
},
{
"epoch": 0.74,
"learning_rate": 3.2448768911481577e-06,
"loss": 0.8351,
"step": 317
},
{
"epoch": 0.75,
"learning_rate": 3.1889845059409552e-06,
"loss": 0.8315,
"step": 318
},
{
"epoch": 0.75,
"learning_rate": 3.1334862234391627e-06,
"loss": 0.7783,
"step": 319
},
{
"epoch": 0.75,
"learning_rate": 3.078385254914764e-06,
"loss": 0.7982,
"step": 320
},
{
"epoch": 0.75,
"learning_rate": 3.023684788650154e-06,
"loss": 0.8098,
"step": 321
},
{
"epoch": 0.76,
"learning_rate": 2.9693879897536436e-06,
"loss": 0.8252,
"step": 322
},
{
"epoch": 0.76,
"learning_rate": 2.91549799997632e-06,
"loss": 0.8619,
"step": 323
},
{
"epoch": 0.76,
"learning_rate": 2.8620179375302604e-06,
"loss": 0.8367,
"step": 324
},
{
"epoch": 0.76,
"learning_rate": 2.8089508969081e-06,
"loss": 0.7489,
"step": 325
},
{
"epoch": 0.77,
"learning_rate": 2.7562999487039822e-06,
"loss": 0.8152,
"step": 326
},
{
"epoch": 0.77,
"learning_rate": 2.7040681394358813e-06,
"loss": 0.8379,
"step": 327
},
{
"epoch": 0.77,
"learning_rate": 2.6522584913693295e-06,
"loss": 0.8188,
"step": 328
},
{
"epoch": 0.77,
"learning_rate": 2.6008740023425248e-06,
"loss": 0.796,
"step": 329
},
{
"epoch": 0.77,
"learning_rate": 2.5499176455928933e-06,
"loss": 0.8382,
"step": 330
},
{
"epoch": 0.78,
"learning_rate": 2.4993923695850307e-06,
"loss": 0.8543,
"step": 331
},
{
"epoch": 0.78,
"learning_rate": 2.449301097840107e-06,
"loss": 0.8014,
"step": 332
},
{
"epoch": 0.78,
"learning_rate": 2.3996467287666914e-06,
"loss": 0.7993,
"step": 333
},
{
"epoch": 0.78,
"learning_rate": 2.3504321354930572e-06,
"loss": 0.7433,
"step": 334
},
{
"epoch": 0.79,
"learning_rate": 2.3016601657009364e-06,
"loss": 0.7676,
"step": 335
},
{
"epoch": 0.79,
"learning_rate": 2.253333641460732e-06,
"loss": 0.784,
"step": 336
},
{
"epoch": 0.79,
"learning_rate": 2.205455359068227e-06,
"loss": 0.7983,
"step": 337
},
{
"epoch": 0.79,
"learning_rate": 2.1580280888828e-06,
"loss": 0.8481,
"step": 338
},
{
"epoch": 0.8,
"learning_rate": 2.1110545751671074e-06,
"loss": 0.8255,
"step": 339
},
{
"epoch": 0.8,
"learning_rate": 2.0645375359283047e-06,
"loss": 0.78,
"step": 340
},
{
"epoch": 0.8,
"learning_rate": 2.0184796627607728e-06,
"loss": 0.7633,
"step": 341
},
{
"epoch": 0.8,
"learning_rate": 1.972883620690366e-06,
"loss": 0.8385,
"step": 342
},
{
"epoch": 0.81,
"learning_rate": 1.9277520480202205e-06,
"loss": 0.7888,
"step": 343
},
{
"epoch": 0.81,
"learning_rate": 1.8830875561780904e-06,
"loss": 0.8004,
"step": 344
},
{
"epoch": 0.81,
"learning_rate": 1.8388927295652449e-06,
"loss": 0.8498,
"step": 345
},
{
"epoch": 0.81,
"learning_rate": 1.795170125406921e-06,
"loss": 0.8315,
"step": 346
},
{
"epoch": 0.81,
"learning_rate": 1.7519222736043662e-06,
"loss": 0.8116,
"step": 347
},
{
"epoch": 0.82,
"learning_rate": 1.7091516765884464e-06,
"loss": 0.8113,
"step": 348
},
{
"epoch": 0.82,
"learning_rate": 1.6668608091748495e-06,
"loss": 0.817,
"step": 349
},
{
"epoch": 0.82,
"learning_rate": 1.625052118420889e-06,
"loss": 0.7449,
"step": 350
},
{
"epoch": 0.82,
"learning_rate": 1.5837280234839013e-06,
"loss": 0.7696,
"step": 351
},
{
"epoch": 0.83,
"learning_rate": 1.542890915481282e-06,
"loss": 0.8255,
"step": 352
},
{
"epoch": 0.83,
"learning_rate": 1.5025431573521209e-06,
"loss": 0.8188,
"step": 353
},
{
"epoch": 0.83,
"learning_rate": 1.4626870837204776e-06,
"loss": 0.8371,
"step": 354
},
{
"epoch": 0.83,
"learning_rate": 1.4233250007602873e-06,
"loss": 0.7561,
"step": 355
},
{
"epoch": 0.84,
"learning_rate": 1.3844591860619382e-06,
"loss": 0.7779,
"step": 356
},
{
"epoch": 0.84,
"learning_rate": 1.346091888500466e-06,
"loss": 0.8593,
"step": 357
},
{
"epoch": 0.84,
"learning_rate": 1.308225328105439e-06,
"loss": 0.7571,
"step": 358
},
{
"epoch": 0.84,
"learning_rate": 1.2708616959324893e-06,
"loss": 0.8681,
"step": 359
},
{
"epoch": 0.85,
"learning_rate": 1.2340031539365483e-06,
"loss": 0.7252,
"step": 360
},
{
"epoch": 0.85,
"learning_rate": 1.1976518348467426e-06,
"loss": 0.822,
"step": 361
},
{
"epoch": 0.85,
"learning_rate": 1.161809842042988e-06,
"loss": 0.8157,
"step": 362
},
{
"epoch": 0.85,
"learning_rate": 1.1264792494342858e-06,
"loss": 0.8135,
"step": 363
},
{
"epoch": 0.85,
"learning_rate": 1.091662101338714e-06,
"loss": 0.8323,
"step": 364
},
{
"epoch": 0.86,
"learning_rate": 1.0573604123651504e-06,
"loss": 0.7858,
"step": 365
},
{
"epoch": 0.86,
"learning_rate": 1.0235761672966903e-06,
"loss": 0.8419,
"step": 366
},
{
"epoch": 0.86,
"learning_rate": 9.903113209758098e-07,
"loss": 0.7914,
"step": 367
},
{
"epoch": 0.86,
"learning_rate": 9.575677981912457e-07,
"loss": 0.8128,
"step": 368
},
{
"epoch": 0.87,
"learning_rate": 9.25347493566634e-07,
"loss": 0.78,
"step": 369
},
{
"epoch": 0.87,
"learning_rate": 8.936522714508678e-07,
"loss": 0.813,
"step": 370
},
{
"epoch": 0.87,
"learning_rate": 8.624839658102346e-07,
"loss": 0.8214,
"step": 371
},
{
"epoch": 0.87,
"learning_rate": 8.318443801222919e-07,
"loss": 0.825,
"step": 372
},
{
"epoch": 0.88,
"learning_rate": 8.017352872715078e-07,
"loss": 0.8359,
"step": 373
},
{
"epoch": 0.88,
"learning_rate": 7.7215842944669e-07,
"loss": 0.7801,
"step": 374
},
{
"epoch": 0.88,
"learning_rate": 7.431155180401705e-07,
"loss": 0.7953,
"step": 375
},
{
"epoch": 0.88,
"learning_rate": 7.146082335487825e-07,
"loss": 0.7873,
"step": 376
},
{
"epoch": 0.88,
"learning_rate": 6.866382254766158e-07,
"loss": 0.861,
"step": 377
},
{
"epoch": 0.89,
"learning_rate": 6.592071122395849e-07,
"loss": 0.7912,
"step": 378
},
{
"epoch": 0.89,
"learning_rate": 6.323164810717753e-07,
"loss": 0.7769,
"step": 379
},
{
"epoch": 0.89,
"learning_rate": 6.059678879336006e-07,
"loss": 0.8244,
"step": 380
},
{
"epoch": 0.89,
"learning_rate": 5.801628574217733e-07,
"loss": 0.8066,
"step": 381
},
{
"epoch": 0.9,
"learning_rate": 5.549028826810888e-07,
"loss": 0.8034,
"step": 382
},
{
"epoch": 0.9,
"learning_rate": 5.301894253180295e-07,
"loss": 0.7885,
"step": 383
},
{
"epoch": 0.9,
"learning_rate": 5.060239153161872e-07,
"loss": 0.8412,
"step": 384
},
{
"epoch": 0.9,
"learning_rate": 4.824077509535252e-07,
"loss": 0.7949,
"step": 385
},
{
"epoch": 0.91,
"learning_rate": 4.5934229872146683e-07,
"loss": 0.7916,
"step": 386
},
{
"epoch": 0.91,
"learning_rate": 4.3682889324583086e-07,
"loss": 0.7595,
"step": 387
},
{
"epoch": 0.91,
"learning_rate": 4.1486883720960436e-07,
"loss": 0.7135,
"step": 388
},
{
"epoch": 0.91,
"learning_rate": 3.934634012775662e-07,
"loss": 0.795,
"step": 389
},
{
"epoch": 0.92,
"learning_rate": 3.726138240227628e-07,
"loss": 0.8281,
"step": 390
},
{
"epoch": 0.92,
"learning_rate": 3.5232131185484075e-07,
"loss": 0.8039,
"step": 391
},
{
"epoch": 0.92,
"learning_rate": 3.325870389502439e-07,
"loss": 0.7719,
"step": 392
},
{
"epoch": 0.92,
"learning_rate": 3.1341214718426885e-07,
"loss": 0.8072,
"step": 393
},
{
"epoch": 0.92,
"learning_rate": 2.9479774606499755e-07,
"loss": 0.8029,
"step": 394
},
{
"epoch": 0.93,
"learning_rate": 2.7674491266909023e-07,
"loss": 0.7855,
"step": 395
},
{
"epoch": 0.93,
"learning_rate": 2.5925469157947135e-07,
"loss": 0.8173,
"step": 396
},
{
"epoch": 0.93,
"learning_rate": 2.423280948248841e-07,
"loss": 0.7435,
"step": 397
},
{
"epoch": 0.93,
"learning_rate": 2.2596610182133328e-07,
"loss": 0.8452,
"step": 398
},
{
"epoch": 0.94,
"learning_rate": 2.1016965931541012e-07,
"loss": 0.7982,
"step": 399
},
{
"epoch": 0.94,
"learning_rate": 1.9493968132951456e-07,
"loss": 0.771,
"step": 400
},
{
"epoch": 0.94,
"learning_rate": 1.802770491089667e-07,
"loss": 0.7866,
"step": 401
},
{
"epoch": 0.94,
"learning_rate": 1.661826110710163e-07,
"loss": 0.8198,
"step": 402
},
{
"epoch": 0.95,
"learning_rate": 1.5265718275574658e-07,
"loss": 0.7537,
"step": 403
},
{
"epoch": 0.95,
"learning_rate": 1.3970154677889313e-07,
"loss": 0.7839,
"step": 404
},
{
"epoch": 0.95,
"learning_rate": 1.2731645278655448e-07,
"loss": 0.7849,
"step": 405
},
{
"epoch": 0.95,
"learning_rate": 1.1550261741181568e-07,
"loss": 0.8754,
"step": 406
},
{
"epoch": 0.96,
"learning_rate": 1.0426072423328382e-07,
"loss": 0.7954,
"step": 407
},
{
"epoch": 0.96,
"learning_rate": 9.359142373553287e-08,
"loss": 0.8272,
"step": 408
},
{
"epoch": 0.96,
"learning_rate": 8.349533327146719e-08,
"loss": 0.7997,
"step": 409
},
{
"epoch": 0.96,
"learning_rate": 7.397303702659675e-08,
"loss": 0.7752,
"step": 410
},
{
"epoch": 0.96,
"learning_rate": 6.50250859852375e-08,
"loss": 0.8586,
"step": 411
},
{
"epoch": 0.97,
"learning_rate": 5.665199789862907e-08,
"loss": 0.8161,
"step": 412
},
{
"epoch": 0.97,
"learning_rate": 4.88542572549755e-08,
"loss": 0.8268,
"step": 413
},
{
"epoch": 0.97,
"learning_rate": 4.163231525141309e-08,
"loss": 0.8432,
"step": 414
},
{
"epoch": 0.97,
"learning_rate": 3.498658976790248e-08,
"loss": 0.8082,
"step": 415
},
{
"epoch": 0.98,
"learning_rate": 2.8917465343047956e-08,
"loss": 0.7384,
"step": 416
},
{
"epoch": 0.98,
"learning_rate": 2.3425293151845274e-08,
"loss": 0.873,
"step": 417
},
{
"epoch": 0.98,
"learning_rate": 1.851039098537122e-08,
"loss": 0.814,
"step": 418
},
{
"epoch": 0.98,
"learning_rate": 1.4173043232380557e-08,
"loss": 0.8339,
"step": 419
},
{
"epoch": 0.99,
"learning_rate": 1.0413500862864745e-08,
"loss": 0.8343,
"step": 420
},
{
"epoch": 0.99,
"learning_rate": 7.231981413520217e-09,
"loss": 0.733,
"step": 421
},
{
"epoch": 0.99,
"learning_rate": 4.628668975166228e-09,
"loss": 0.817,
"step": 422
},
{
"epoch": 0.99,
"learning_rate": 2.6037141820933752e-09,
"loss": 0.8674,
"step": 423
},
{
"epoch": 1.0,
"learning_rate": 1.157234203341684e-09,
"loss": 0.8376,
"step": 424
},
{
"epoch": 1.0,
"learning_rate": 2.8931273592824884e-10,
"loss": 0.803,
"step": 425
},
{
"epoch": 1.0,
"learning_rate": 0.0,
"loss": 0.8391,
"step": 426
},
{
"epoch": 1.0,
"step": 426,
"total_flos": 1.7489761962148495e+18,
"train_loss": 0.8527796469943624,
"train_runtime": 1012.9291,
"train_samples_per_second": 214.873,
"train_steps_per_second": 0.421
}
],
"logging_steps": 1.0,
"max_steps": 426,
"num_train_epochs": 1,
"save_steps": 5000,
"total_flos": 1.7489761962148495e+18,
"trial_name": null,
"trial_params": null
}
cache/models--openai--clip-vit-large-patch14/README.md 0 → 100644
View file @ c873301f
---
tags:
- vision
widget:
- src: https://huggingface.co/datasets/mishig/sample_images/resolve/main/cat-dog-music.png
candidate_labels: playing music, playing sports
example_title: Cat & Dog
---
# Model Card: CLIP
Disclaimer: The model card is taken and modified from the official CLIP repository, it can be found [here](https://github.com/openai/CLIP/blob/main/model-card.md).
## Model Details
The CLIP model was developed by researchers at OpenAI to learn about what contributes to robustness in computer vision tasks. The model was also developed to test the ability of models to generalize to arbitrary image classification tasks in a zero-shot manner. It was not developed for general model deployment - to deploy models like CLIP, researchers will first need to carefully study their capabilities in relation to the specific context they’re being deployed within.
### Model Date
January 2021
### Model Type
The base model uses a ViT-L/14 Transformer architecture as an image encoder and uses a masked self-attention Transformer as a text encoder. These encoders are trained to maximize the similarity of (image, text) pairs via a contrastive loss.
The original implementation had two variants: one using a ResNet image encoder and the other using a Vision Transformer. This repository has the variant with the Vision Transformer.
### Documents
- [Blog Post](https://openai.com/blog/clip/)
- [CLIP Paper](https://arxiv.org/abs/2103.00020)
### Use with Transformers
```python
from PIL import Image
import requests
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained("openai/clip-vit-large-patch14")
processor = CLIPProcessor.from_pretrained("openai/clip-vit-large-patch14")
url = "http://images.cocodataset.org/val2017/000000039769.jpg"
image = Image.open(requests.get(url, stream=True).raw)
inputs = processor(text=["a photo of a cat", "a photo of a dog"], images=image, return_tensors="pt", padding=True)
outputs = model(**inputs)
logits_per_image = outputs.logits_per_image # this is the image-text similarity score
probs = logits_per_image.softmax(dim=1) # we can take the softmax to get the label probabilities
```
## Model Use
### Intended Use
The model is intended as a research output for research communities. We hope that this model will enable researchers to better understand and explore zero-shot, arbitrary image classification. We also hope it can be used for interdisciplinary studies of the potential impact of such models - the CLIP paper includes a discussion of potential downstream impacts to provide an example for this sort of analysis.
#### Primary intended uses
The primary intended users of these models are AI researchers.
We primarily imagine the model will be used by researchers to better understand robustness, generalization, and other capabilities, biases, and constraints of computer vision models.
### Out-of-Scope Use Cases
**Any** deployed use case of the model - whether commercial or not - is currently out of scope. Non-deployed use cases such as image search in a constrained environment, are also not recommended unless there is thorough in-domain testing of the model with a specific, fixed class taxonomy. This is because our safety assessment demonstrated a high need for task specific testing especially given the variability of CLIP’s performance with different class taxonomies. This makes untested and unconstrained deployment of the model in any use case currently potentially harmful.
Certain use cases which would fall under the domain of surveillance and facial recognition are always out-of-scope regardless of performance of the model. This is because the use of artificial intelligence for tasks such as these can be premature currently given the lack of testing norms and checks to ensure its fair use.
Since the model has not been purposefully trained in or evaluated on any languages other than English, its use should be limited to English language use cases.
## Data
The model was trained on publicly available image-caption data. This was done through a combination of crawling a handful of websites and using commonly-used pre-existing image datasets such as [YFCC100M](http://projects.dfki.uni-kl.de/yfcc100m/). A large portion of the data comes from our crawling of the internet. This means that the data is more representative of people and societies most connected to the internet which tend to skew towards more developed nations, and younger, male users.
### Data Mission Statement
Our goal with building this dataset was to test out robustness and generalizability in computer vision tasks. As a result, the focus was on gathering large quantities of data from different publicly-available internet data sources. The data was gathered in a mostly non-interventionist manner. However, we only crawled websites that had policies against excessively violent and adult images and allowed us to filter out such content. We do not intend for this dataset to be used as the basis for any commercial or deployed model and will not be releasing the dataset.
## Performance and Limitations
### Performance
We have evaluated the performance of CLIP on a wide range of benchmarks across a variety of computer vision datasets such as OCR to texture recognition to fine-grained classification. The paper describes model performance on the following datasets:
- Food101
- CIFAR10
- CIFAR100
- Birdsnap
- SUN397
- Stanford Cars
- FGVC Aircraft
- VOC2007
- DTD
- Oxford-IIIT Pet dataset
- Caltech101
- Flowers102
- MNIST
- SVHN
- IIIT5K
- Hateful Memes
- SST-2
- UCF101
- Kinetics700
- Country211
- CLEVR Counting
- KITTI Distance
- STL-10
- RareAct
- Flickr30
- MSCOCO
- ImageNet
- ImageNet-A
- ImageNet-R
- ImageNet Sketch
- ObjectNet (ImageNet Overlap)
- Youtube-BB
- ImageNet-Vid
## Limitations
CLIP and our analysis of it have a number of limitations. CLIP currently struggles with respect to certain tasks such as fine grained classification and counting objects. CLIP also poses issues with regards to fairness and bias which we discuss in the paper and briefly in the next section. Additionally, our approach to testing CLIP also has an important limitation- in many cases we have used linear probes to evaluate the performance of CLIP and there is evidence suggesting that linear probes can underestimate model performance.
### Bias and Fairness
We find that the performance of CLIP - and the specific biases it exhibits - can depend significantly on class design and the choices one makes for categories to include and exclude. We tested the risk of certain kinds of denigration with CLIP by classifying images of people from [Fairface](https://arxiv.org/abs/1908.04913) into crime-related and non-human animal categories. We found significant disparities with respect to race and gender. Additionally, we found that these disparities could shift based on how the classes were constructed. (Details captured in the Broader Impacts Section in the paper).
We also tested the performance of CLIP on gender, race and age classification using the Fairface dataset (We default to using race categories as they are constructed in the Fairface dataset.) in order to assess quality of performance across different demographics. We found accuracy >96% across all races for gender classification with ‘Middle Eastern’ having the highest accuracy (98.4%) and ‘White’ having the lowest (96.5%). Additionally, CLIP averaged ~93% for racial classification and ~63% for age classification. Our use of evaluations to test for gender, race and age classification as well as denigration harms is simply to evaluate performance of the model across people and surface potential risks and not to demonstrate an endorsement/enthusiasm for such tasks.
## Feedback
### Where to send questions or comments about the model
Please use [this Google Form](https://forms.gle/Uv7afRH5dvY34ZEs9)
\ No newline at end of file
cache/models--openai--clip-vit-large-patch14/config.json 0 → 100644
View file @ c873301f
{
"_name_or_path": "clip-vit-large-patch14/",
"architectures": [
"CLIPModel"
],
"initializer_factor": 1.0,
"logit_scale_init_value": 2.6592,
"model_type": "clip",
"projection_dim": 768,
"text_config": {
"_name_or_path": "",
"add_cross_attention": false,
"architectures": null,
"attention_dropout": 0.0,
"bad_words_ids": null,
"bos_token_id": 0,
"chunk_size_feed_forward": 0,
"cross_attention_hidden_size": null,
"decoder_start_token_id": null,
"diversity_penalty": 0.0,
"do_sample": false,
"dropout": 0.0,
"early_stopping": false,
"encoder_no_repeat_ngram_size": 0,
"eos_token_id": 2,
"finetuning_task": null,
"forced_bos_token_id": null,
"forced_eos_token_id": null,
"hidden_act": "quick_gelu",
"hidden_size": 768,
"id2label": {
"0": "LABEL_0",
"1": "LABEL_1"
},
"initializer_factor": 1.0,
"initializer_range": 0.02,
"intermediate_size": 3072,
"is_decoder": false,
"is_encoder_decoder": false,
"label2id": {
"LABEL_0": 0,
"LABEL_1": 1
},
"layer_norm_eps": 1e-05,
"length_penalty": 1.0,
"max_length": 20,
"max_position_embeddings": 77,
"min_length": 0,
"model_type": "clip_text_model",
"no_repeat_ngram_size": 0,
"num_attention_heads": 12,
"num_beam_groups": 1,
"num_beams": 1,
"num_hidden_layers": 12,
"num_return_sequences": 1,
"output_attentions": false,
"output_hidden_states": false,
"output_scores": false,
"pad_token_id": 1,
"prefix": null,
"problem_type": null,
"projection_dim" : 768,
"pruned_heads": {},
"remove_invalid_values": false,
"repetition_penalty": 1.0,
"return_dict": true,
"return_dict_in_generate": false,
"sep_token_id": null,
"task_specific_params": null,
"temperature": 1.0,
"tie_encoder_decoder": false,
"tie_word_embeddings": true,
"tokenizer_class": null,
"top_k": 50,
"top_p": 1.0,
"torch_dtype": null,
"torchscript": false,
"transformers_version": "4.16.0.dev0",
"use_bfloat16": false,
"vocab_size": 49408
},
"text_config_dict": {
"hidden_size": 768,
"intermediate_size": 3072,
"num_attention_heads": 12,
"num_hidden_layers": 12,
"projection_dim": 768
},
"torch_dtype": "float32",
"transformers_version": null,
"vision_config": {
"_name_or_path": "",
"add_cross_attention": false,
"architectures": null,
"attention_dropout": 0.0,
"bad_words_ids": null,
"bos_token_id": null,
"chunk_size_feed_forward": 0,
"cross_attention_hidden_size": null,
"decoder_start_token_id": null,
"diversity_penalty": 0.0,
"do_sample": false,
"dropout": 0.0,
"early_stopping": false,
"encoder_no_repeat_ngram_size": 0,
"eos_token_id": null,
"finetuning_task": null,
"forced_bos_token_id": null,
"forced_eos_token_id": null,
"hidden_act": "quick_gelu",
"hidden_size": 1024,
"id2label": {
"0": "LABEL_0",
"1": "LABEL_1"
},
"image_size": 224,
"initializer_factor": 1.0,
"initializer_range": 0.02,
"intermediate_size": 4096,
"is_decoder": false,
"is_encoder_decoder": false,
"label2id": {
"LABEL_0": 0,
"LABEL_1": 1
},
"layer_norm_eps": 1e-05,
"length_penalty": 1.0,
"max_length": 20,
"min_length": 0,
"model_type": "clip_vision_model",
"no_repeat_ngram_size": 0,
"num_attention_heads": 16,
"num_beam_groups": 1,
"num_beams": 1,
"num_hidden_layers": 24,
"num_return_sequences": 1,
"output_attentions": false,
"output_hidden_states": false,
"output_scores": false,
"pad_token_id": null,
"patch_size": 14,
"prefix": null,
"problem_type": null,
"projection_dim" : 768,
"pruned_heads": {},
"remove_invalid_values": false,
"repetition_penalty": 1.0,
"return_dict": true,
"return_dict_in_generate": false,
"sep_token_id": null,
"task_specific_params": null,
"temperature": 1.0,
"tie_encoder_decoder": false,
"tie_word_embeddings": true,
"tokenizer_class": null,
"top_k": 50,
"top_p": 1.0,
"torch_dtype": null,
"torchscript": false,
"transformers_version": "4.16.0.dev0",
"use_bfloat16": false
},
"vision_config_dict": {
"hidden_size": 1024,
"intermediate_size": 4096,
"num_attention_heads": 16,
"num_hidden_layers": 24,
"patch_size": 14,
"projection_dim": 768
}
}
cache/models--openai--clip-vit-large-patch14/gitattributes 0 → 100644
View file @ c873301f
*.7z filter=lfs diff=lfs merge=lfs -text
*.arrow filter=lfs diff=lfs merge=lfs -text
*.bin filter=lfs diff=lfs merge=lfs -text
*.bin.* filter=lfs diff=lfs merge=lfs -text
*.bz2 filter=lfs diff=lfs merge=lfs -text
*.ftz filter=lfs diff=lfs merge=lfs -text
*.gz filter=lfs diff=lfs merge=lfs -text
*.h5 filter=lfs diff=lfs merge=lfs -text
*.joblib filter=lfs diff=lfs merge=lfs -text
*.lfs.* filter=lfs diff=lfs merge=lfs -text
*.model filter=lfs diff=lfs merge=lfs -text
*.msgpack filter=lfs diff=lfs merge=lfs -text
*.onnx filter=lfs diff=lfs merge=lfs -text
*.ot filter=lfs diff=lfs merge=lfs -text
*.parquet filter=lfs diff=lfs merge=lfs -text
*.pb filter=lfs diff=lfs merge=lfs -text
*.pt filter=lfs diff=lfs merge=lfs -text
*.pth filter=lfs diff=lfs merge=lfs -text
*.rar filter=lfs diff=lfs merge=lfs -text
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
*.tar.* filter=lfs diff=lfs merge=lfs -text
*.tflite filter=lfs diff=lfs merge=lfs -text
*.tgz filter=lfs diff=lfs merge=lfs -text
*.xz filter=lfs diff=lfs merge=lfs -text
*.zip filter=lfs diff=lfs merge=lfs -text
*.zstandard filter=lfs diff=lfs merge=lfs -text
*tfevents* filter=lfs diff=lfs merge=lfs -text
model.safetensors filter=lfs diff=lfs merge=lfs -text
cache/models--openai--clip-vit-large-patch14/merges.txt 0 → 100644
View file @ c873301f
This source diff could not be displayed because it is too large. You can view the blob instead.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment