Commits · d73c2b1d79ebd20e0a9c8947cc18da31e559d32d · OpenDAS / ColossalAI

"tests/vscode:/vscode.git/clone" did not exist on "5156d5b4f8bdf3f402b7a1fd5786eff2d2266c8b"

11 May, 2022 2 commits

[Tensor] fix init context (#931) · d73c2b1d

Ziyue Jiang authored May 11, 2022

* change torch.Parameter to ColoParameter

* fix post assignment for init context

* polish

* polish

d73c2b1d

[Tensor] simplify named param (#928) · dfc88b85
Ziyue Jiang authored May 11, 2022
```
* simplify ColoModulize

* simplify ColoModulize

* polish

* polish
```
dfc88b85

09 May, 2022 3 commits
- [tensor] hijack addmm for colo tensor (#923) · 45b9124d
  ver217 authored May 09, 2022
```
* hijack addmm for colo tensor

* fix bugs

* polish unit test

* polish comments
```
  45b9124d
- test pretrain loading on multi-process (#922) · 534afb01
  Jiarui Fang authored May 09, 2022
  
  534afb01
- [Tensor] add from_pretrained support and bert pretrained test (#921) · c195d281
  Ziyue Jiang authored May 09, 2022
```
* add from_pretrained support and test

* polish

* polish

* polish

* polish
```
  c195d281
07 May, 2022 2 commits
- [Graph] building computing graph with ColoTensor, Linear only (#917) · 845856ea
  Jiarui Fang authored May 07, 2022
  
  845856ea
- [Tensor] add 1d vocab loss (#918) · 75d22191
  Ziyue Jiang authored May 07, 2022
```
* add 1d vocab loss

* polish
```
  75d22191
06 May, 2022 5 commits
- [Tensor] fix test_model (#916) · dfaff4e2
  Ziyue Jiang authored May 06, 2022
```
* polish test_model

* polish
```
  dfaff4e2
- [Tensor] polish model test (#915) · ed6426c3
  Jiarui Fang authored May 06, 2022
  
  ed6426c3
- [Tensor] add a basic bert. (#911) · 0fab86b1
  Ziyue Jiang authored May 06, 2022
```
* add base bert test

* Add bert test

* polish

* remove test_bert

* polish
```
  0fab86b1
- [Tensor] init ColoParameter (#914) · ab95ec9a
  Jiarui Fang authored May 06, 2022
  
  ab95ec9a
- update pytest.mark.parametrize in tensor tests (#913) · 193d6293
  Ziyue Jiang authored May 06, 2022
  
  193d6293
29 Apr, 2022 1 commit
- [Tensor] add embedding tp1d row (#904) · f593a563
  Ziyue Jiang authored Apr 29, 2022
  
  f593a563
28 Apr, 2022 4 commits
- [Tensor] add ColoTensor TP1Dcol Embedding (#899) · 2c0d19d7
  Ziyue Jiang authored Apr 28, 2022
  
  2c0d19d7
- [Tensor] initialize the ColoOptimizer (#898) · d16671da
  Jiarui Fang authored Apr 28, 2022
```
* [Tensor] activation is an attr of ColoTensor

* [Tensor] add optimizer

* only detach parameters in context

* polish code
```
  d16671da
- [Tensor] test parameters() as member function (#896) · e76f76c0
  Jiarui Fang authored Apr 28, 2022
  
  e76f76c0
- [tensor] refine linear and add gather for laynorm (#893) · cb182da7
  Ziyue Jiang authored Apr 28, 2022
```
* refine linear and add function to ColoTensor

* add gather for layernorm

* polish

* polish
```
  cb182da7
27 Apr, 2022 4 commits
- [Tensor] overriding paramters() for Module using ColoTensor (#889) · 26c49639
  Jiarui Fang authored Apr 27, 2022
  
  26c49639
- [tensor] add ColoTensor 1Dcol (#888) · 1d0aba41
  Ziyue Jiang authored Apr 27, 2022
  
  1d0aba41
- [Tensor] test model check results for a simple net (#887) · a0e59716
  Jiarui Fang authored Apr 27, 2022
  
  a0e59716
- [Tensor] make ColoTensor more robust for getattr (#886) · 72cdc068
  Jiarui Fang authored Apr 27, 2022
```
* [Tensor] make ColoTensor more robust for getattr

* polish

* polish
```
  72cdc068
26 Apr, 2022 7 commits
- [tensor] wrap function in the torch_tensor to ColoTensor (#881) · 9bc5a77c
  Ziyue Jiang authored Apr 26, 2022
  
  9bc5a77c
- [Tensor] make a simple net works with 1D row TP (#879) · 7f76517a
  Jiarui Fang authored Apr 26, 2022
  
  7f76517a
- [gemini] accelerate adjust_layout() (#878) · c4d903e6
  ver217 authored Apr 26, 2022
```
* add lru cache

* polish code

* update unit test

* fix sharded optim
```
  c4d903e6
- [Tensor] Add some attributes to ColoTensor (#877) · 90921145
  Jiarui Fang authored Apr 26, 2022
```
* [Tensor] add some function to ColoTensor

* torch.allclose

* rm torch.add
```
  90921145
- [Tensor] get named parameters for model using ColoTensors (#874) · e43f83aa
  Jiarui Fang authored Apr 26, 2022
  
  e43f83aa
- [tensor] customized op returns ColoTensor (#875) · 96211c2c
  Jiarui Fang authored Apr 26, 2022
```
* [tensor] customized op returns ColoTensor

* polish

* polish code
```
  96211c2c
- [Tensor] Add function to spec and update linear 1Drow and unit tests (#869) · 26d4ab8b
  Ziyue Jiang authored Apr 26, 2022
  
  26d4ab8b
25 Apr, 2022 5 commits
- [tensor] add cross_entrophy_loss (#868) · 1190b2c4
  Jiarui Fang authored Apr 25, 2022
  
  1190b2c4
- [gemini] add stateful tensor container (#867) · 31078171
  HELSON authored Apr 25, 2022
  
  31078171
- colo init context add device attr. (#866) · d01d3b8c
  Jiarui Fang authored Apr 25, 2022
  
  d01d3b8c
- [Tensor] add layer norm Op (#852) · 126ba573
  Jiarui Fang authored Apr 25, 2022
  
  126ba573
- [ci] cache cuda extension (#860) · 1258af71
  Frank Lee authored Apr 25, 2022
  
  1258af71
24 Apr, 2022 7 commits
- [Tensor ] Add 1Drow weight reshard by spec (#854) · bcc86550
  Ziyue Jiang authored Apr 24, 2022
  
  bcc86550
- [Tensor] init a tp network training unittest (#849) · 62f05925
  Jiarui Fang authored Apr 24, 2022
  
  62f05925
- [tensor]add assert for colo_tensor 1Drow (#846) · 2a0a427e
  Ziyue Jiang authored Apr 24, 2022
  
  2a0a427e
- [Tensor] TP Linear 1D row (#843) · 05023ecf
  Ziyue Jiang authored Apr 24, 2022
  
  05023ecf
- [gemini] add GeminiMemoryManger (#832) · e5ea3fde
  HELSON authored Apr 24, 2022
```
* refactor StatefulTensor, tensor utilities

* add unitest for GeminiMemoryManager
```
  e5ea3fde
- [pipelinable]use pipelinable context to initialize non-pipeline model (#816) · 35ea6e10
  YuliangLiu0306 authored Apr 24, 2022
```
* [CLI] add CLI launcher

* Revert "[CLI] add CLI launcher"

This reverts commit df7e6506d4500af6a9220ef7fe4d3c7b1daebd4c.

* [pipeline]add module lazy init feature to support large model initization.

* [pipeline]add to_layer_list and partition method to support arbitrary non-pp model

* refactor the module structure

* polish

* [pipelinable]add unit test for pipelinable

* polish

* polish

* Fix CodeFactor issues.
```
  35ea6e10
- [hotfix] the bug of numel() in ColoTensor (#845) · ea0a2ed2
  Jiarui Fang authored Apr 24, 2022
  
  ea0a2ed2