Unverified Commit 33268d8c authored by echo840's avatar echo840 Committed by GitHub
Browse files

Update README.md

parent 4747dc52
...@@ -78,12 +78,6 @@ We also offer Monkey's model definition and training code, which you can explore ...@@ -78,12 +78,6 @@ We also offer Monkey's model definition and training code, which you can explore
The json file used for Monkey training can be downloaded at [Link](https://drive.google.com/file/d/18z_uQTe8Jq61V5rgHtxOt85uKBodbvw1/view?usp=sharing). The json file used for Monkey training can be downloaded at [Link](https://drive.google.com/file/d/18z_uQTe8Jq61V5rgHtxOt85uKBodbvw1/view?usp=sharing).
**ATTENTION:** Specify the path to your training data, which should be a json file consisting of a list of conversations.
Inspired by Qwen-VL, we freeze the Large Language Model (LLM) and introduce LoRA into four linear layers ```"c_attn", "attn.c_proj", "w1", "w2"``` for training. This step makes it possible to train Monkey using 8 NVIDIA 3090 GPUs. The specific implementation code is in ```modeling_qwen_nvdia3090.py```.
- Add LoRA: You need to replace the contents of ```modeling_qwen.py``` with the contents of ```modeling_qwen_nvdia3090.py```.
- Freeze LLM: You need to freeze other modules except LoRA and Resampler modules in ```finetune_multitask.py```.
## Inference ## Inference
Run the inference code for Monkey and Monkey-Chat: Run the inference code for Monkey and Monkey-Chat:
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment