Commits · 32cb74493a7a1554ab6b7475f57e4ec9e648ae84 · OpenDAS / ColossalAI

18 Jan, 2024 1 commit
- fix auto loading gpt2 tokenizer (#5279) · 32cb7449
  Michelle authored Jan 18, 2024
  
  32cb7449
17 Jan, 2024 3 commits
- [hotfix] Fix ShardFormer test execution path when using sequence parallelism (#5230) · 5d9a0ae7
  Zhongkai Zhao authored Jan 17, 2024
  
  5d9a0ae7
- [shardformer] hybridparallelplugin support gradients accumulation. (#5246) · 46e09165
  flybird11111 authored Jan 17, 2024
```
* support gradients acc

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

* fix

fix

* fix

fix

fix
```
  46e09165
- [ci] fix test_hybrid_parallel_plugin_checkpoint_io.py (#5276) · 2a0558d8
  flybird11111 authored Jan 17, 2024
```
* fix ci

fix

* fix test

* revert: revert p2p

* feat: add enable_metadata_cache option

* revert: enable t5 tests

* fix

---------
Co-authored-by: Wenhao Chen <cwher@outlook.com>
```
  2a0558d8
16 Jan, 2024 2 commits
- [workflow] fixed oom tests (#5275) · d69cd2eb
  Frank Lee authored Jan 16, 2024
```
* [workflow] fixed oom tests

* polish

* polish

* polish
```
  d69cd2eb
- [workflow] fixed incomplete bash command (#5272) · 04244aaa
  Frank Lee authored Jan 16, 2024
  
  04244aaa
15 Jan, 2024 1 commit
- [hotfix]: add pp sanity check and fix mbs arg (#5268) · ef4f0ee8
  Wenhao Chen authored Jan 15, 2024
```
* fix: fix misleading mbs arg

* feat: add pp sanity check

* fix: fix 1f1b sanity check
```
  ef4f0ee8
11 Jan, 2024 5 commits
- [doc] fix doc typo (#5256) · c174c4fc
  binmakeswell authored Jan 11, 2024
```
* [doc] fix annotation display

* [doc] fix llama2 doc
```
  c174c4fc
- [ci] fix shardformer tests. (#5255) · e830ef91
  flybird11111 authored Jan 11, 2024
```
* fix ci

fix

* revert: revert p2p

* feat: add enable_metadata_cache option

* revert: enable t5 tests

---------
Co-authored-by: Wenhao Chen <cwher@outlook.com>
```
  e830ef91
- fix typo in applications/ColossalEval/README.md (#5250) · 756c400a
  digger yu authored Jan 11, 2024
  
  756c400a
- [ci] fixed ddp test (#5254) · 2b834187
  Frank Lee authored Jan 11, 2024
```
* [ci] fixed ddp test

* polish
```
  2b834187
- [ci] fixed booster test (#5251) · d5eeeb14
  Frank Lee authored Jan 11, 2024
```
* [ci] fixed booster test

* [ci] fixed booster test

* [ci] fixed booster test
```
  d5eeeb14
10 Jan, 2024 2 commits
- [workflow] fixed build CI (#5240) · edf94a35
  Frank Lee authored Jan 10, 2024
```
* [workflow] fixed build CI

* polish

* polish

* polish

* polish

* polish
```
  edf94a35
- [doc] fix typo in Colossal-LLaMA-2/README.md (#5247) · 41e52c1c
  digger yu authored Jan 10, 2024
  
  41e52c1c
08 Jan, 2024 2 commits

[pipeline] A more general _communicate in p2p (#5062) · d565df38

Elsa Granger authored Jan 08, 2024



* A more general _communicate

* feat: finish tree_flatten version p2p

* fix: update p2p api calls

---------
Co-authored-by: Wenhao Chen <cwher@outlook.com>

d565df38

[doc] SwiftInfer release (#5236) · 7bc6969c

binmakeswell authored Jan 08, 2024

* [doc] SwiftInfer release

* [doc] SwiftInfer release

* [doc] SwiftInfer release

* [doc] SwiftInfer release

* [doc] SwiftInfer release

7bc6969c

07 Jan, 2024 2 commits
- [format] applied code formatting on changed files in pull request 5234 (#5235) · 4fb4a22a
  github-actions[bot] authored Jan 07, 2024
```
Co-authored-by: github-actions <github-actions@github.com>
```
  4fb4a22a
- [doc] add Colossal-LLaMA-2-13B (#5234) · b9b32b15
  binmakeswell authored Jan 07, 2024
```
* [doc] add Colossal-LLaMA-2-13B

* [doc] add Colossal-LLaMA-2-13B

* [doc] add Colossal-LLaMA-2-13B
```
  b9b32b15
06 Jan, 2024 2 commits
- [doc] Make leaderboard format more uniform and good-looking (#5231) · ce651270
  JIMMY ZHAO authored Jan 06, 2024
```
* Make leaderboard format more unifeid and good-looking

* Update README.md

* Update README.md
```
  ce651270
- [doc] Update README.md of Colossal-LLAMA2 (#5233) · 915b4652
  Camille Zhong authored Jan 06, 2024
```
* Update README.md

* Update README.md
```
  915b4652
05 Jan, 2024 1 commit

[Colossal-LLaMA-2] Release Colossal-LLaMA-2-13b-base model (#5224) · d992b559

Tong Li authored Jan 05, 2024

* update readme

* update readme

* update link

* update

* update readme

* update

* update

* update

* update title

* update example

* update example

* fix content

* add conclusion

* add license

* update

* update

* update version

* fix minor

d992b559

04 Jan, 2024 1 commit
- [nfc] fix typo colossalai/shardformer/ (#5133) · b0b53a17
  digger yu authored Jan 04, 2024
  
  b0b53a17
03 Jan, 2024 4 commits
- fix flash attn (#5209) · 451e9142
  flybird11111 authored Jan 03, 2024
  
  451e9142
- fix-test (#5210) · 365671be
  flybird11111 authored Jan 03, 2024
```
fix-test

fix-test
```
  365671be
- [devops] update torch versoin in ci (#5217) · 7f3400b5
  Hongxin Liu authored Jan 03, 2024
  
  7f3400b5
- [pipeline]: add p2p fallback order and fix interleaved pp deadlock (#5214) · d799a308
  Wenhao Chen authored Jan 03, 2024
```
* fix: add fallback order option and update 1f1b

* fix: fix deadlock comm in interleaved pp

* test: modify p2p test
```
  d799a308
02 Jan, 2024 1 commit

[pipeline]: support arbitrary batch size in forward_only mode (#5201) · 3c0d82b1

Wenhao Chen authored Jan 02, 2024

* fix: remove drop last in val & test dataloader

* feat: add run_forward_only, support arbitrary bs

* chore: modify ci script

3c0d82b1

29 Dec, 2023 1 commit
- support linear accumulation fusion (#5199) · 02d2328a
  flybird11111 authored Dec 29, 2023
```
support linear accumulation fusion

support linear accumulation fusion

fix
```
  02d2328a
27 Dec, 2023 1 commit
- [doc] Update required third-party library list for testing and torch comptibility checking (#5207) · 64519eb8
  Zhongkai Zhao authored Dec 27, 2023
```
* doc/update requirements-test.txt

* update torch-cuda compatibility check
```
  64519eb8
22 Dec, 2023 2 commits

Improve logic for selecting metrics (#5196) · eae01b67
Yuanchen authored Dec 22, 2023
```
Co-authored-by: Xu <yuanchen.xu00@gmail.com>
```
eae01b67

[pipeline]: fix p2p comm, add metadata cache and support llama interleaved pp (#5134) · 4fa689fc

Wenhao Chen authored Dec 22, 2023

* test: add more p2p tests

* fix: remove send_forward_recv_forward as p2p op list need to use the same group

* fix: make send and receive atomic

* feat: update P2PComm fn

* feat: add metadata cache in 1f1b

* feat: add metadata cache in interleaved pp

* feat: modify is_xx_stage fn

* revert: add _broadcast_object_list

* feat: add interleaved pp in llama policy

* feat: set NCCL_BUFFSIZE in HybridParallelPlugin

4fa689fc

20 Dec, 2023 1 commit
- polish readme in application/chat (#5194) · af952673
  BlueRum authored Dec 20, 2023
  
  af952673
15 Dec, 2023 2 commits
- [doc] update pytorch version in documents. (#5177) · 681d9b12
  flybird11111 authored Dec 15, 2023
```
* fix

aaa

fix

fix

fix

* fix

* fix

* test ci

* fix ci

fix

* update pytorch version in documents
```
  681d9b12
- Fix ColossalEval (#5186) · 3ff60d13
  Yuanchen authored Dec 15, 2023
```
Co-authored-by: Xu Yuanchen <yuanchen.xu00@gmail.com>
```
  3ff60d13
12 Dec, 2023 2 commits

[shardformer] llama support DistCrossEntropy (#5176) · 79718fae

flybird11111 authored Dec 13, 2023



* fix

aaa

fix

fix

fix

* fix

* fix

* test ci

* fix ci

fix

* llama support dist-cross

fix

fix

fix

fix

fix

fix

fix

fix

* fix

* fix

* fix

fix

* test ci

* test ci

* fix

* [Colossal-Llama-2] Add finetuning Colossal-Llama-2 example (#4878)

* Add finetuning Colossal-Llama-2 example

* Add finetuning Colossal-Llama-2 example 2

* Add finetuning Colossal-Llama-2 example and support NEFTuning

* Add inference example and refine neftune

* Modify readme file

* update the imports

---------
Co-authored-by: Xu Yuanchen <yuanchen.xu00@gmail.com>
Co-authored-by: Camille Zhong <44392324+Camille7777@users.noreply.github.com>

* llama support dist-cross

fix

fix

fix

fix

fix

fix

fix

fix

* fix

* fix

* fix

fix

* test ci

* test ci

* fix

* fix ci

* fix ci

---------
Co-authored-by: Yuanchen <70520919+chengeharrison@users.noreply.github.com>
Co-authored-by: Xu Yuanchen <yuanchen.xu00@gmail.com>
Co-authored-by: Camille Zhong <44392324+Camille7777@users.noreply.github.com>

79718fae

[ColossalEval] Support GSM, Data Leakage Evaluation and Tensor Parallel (#5169) · cefdc326

Yuanchen authored Dec 12, 2023



* Support GSM, Data Leakage Evaluation and Tensor Parallel

* remove redundant code and update inference.py in examples/gpt_evaluation

---------
Co-authored-by: Xu Yuanchen <yuanchen.xu00@gmail.com>

cefdc326

11 Dec, 2023 1 commit
- [colossalqa] fix pangu api (#5170) · b07a6f4e
  Michelle authored Dec 11, 2023
```
* fix pangu api

* add comment
```
  b07a6f4e
08 Dec, 2023 1 commit
- [gemini] hotfix NaN loss while using Gemini + tensor_parallel (#5150) · 21aa5de0
  flybird11111 authored Dec 08, 2023
```
* fix

aaa

fix

fix

fix

* fix

* fix

* test ci

* fix ci

fix
```
  21aa5de0
07 Dec, 2023 1 commit

[Colossal-Llama-2] Add finetuning Colossal-Llama-2 example (#4878) · b3971044

Yuanchen authored Dec 07, 2023



* Add finetuning Colossal-Llama-2 example

* Add finetuning Colossal-Llama-2 example 2

* Add finetuning Colossal-Llama-2 example and support NEFTuning

* Add inference example and refine neftune

* Modify readme file

* update the imports

---------
Co-authored-by: Xu Yuanchen <yuanchen.xu00@gmail.com>
Co-authored-by: Camille Zhong <44392324+Camille7777@users.noreply.github.com>

b3971044

05 Dec, 2023 1 commit
- fix (#5158) · 3dbbf83f
  flybird11111 authored Dec 05, 2023
```
fix
```
  3dbbf83f