Commits · a2a_overlap · evt_fugx1 / dcu_megatron

12 Jun, 2025 2 commits
- rewrite combined_1f1b · bb6ab0fb
  dongcl authored Jun 12, 2025
  
  bb6ab0fb
- dualpipev support moe a2a overlap · 69add73b
  dongcl authored Jun 12, 2025
  
  69add73b
10 Jun, 2025 1 commit
- evaluate support dualpipev · 62f16817
  dongcl authored Jun 10, 2025
  
  62f16817
07 Jun, 2025 1 commit
- modify dualpipev, add foward_step_helper and backward_step_helper · 2385a133
  dongcl authored Jun 07, 2025
  
  2385a133
05 Jun, 2025 1 commit
- rewrite dualpipev_schedules; deduplicate the code · cb1230db
  dongcl authored Jun 05, 2025
  
  cb1230db
04 Jun, 2025 1 commit
- if forward_only is true, recv_backward should not be called · a58a2da6
  dongcl authored Jun 04, 2025
  
  a58a2da6
03 Jun, 2025 2 commits
- dualpipev supports evaluation mode · 2b81ee55
  dongcl authored Jun 03, 2025
  
  2b81ee55
- fix dualpipev broadcast error · f3434cc7
  dongcl authored Jun 03, 2025
  
  f3434cc7
30 May, 2025 1 commit
- add feature manager to support feature selection · 7c63d1a4
  dongcl authored May 30, 2025
  
  7c63d1a4
29 May, 2025 2 commits
- add dualpipev_chunks to support dualpipev · 8d5bae2a
  dongcl authored May 29, 2025
  
  8d5bae2a
- Merge branch 'a2a_overlap' of... · e5f5eb4d
  dongcl authored May 29, 2025
```
Merge branch 'a2a_overlap' of http://developer.sourcefind.cn/codes/OpenDAS/dcu_megatron into a2a_overlap
```
  e5f5eb4d
27 May, 2025 2 commits
- add dualpipev · ff5427cf
  dongcl authored May 27, 2025
  
  ff5427cf
- remove unused import · 3947aa6c
  dongcl authored May 27, 2025
  
  3947aa6c
26 May, 2025 2 commits
- 1f1b overlap only supports MoEAlltoAllTokenDispatcher · 7a088067
  dongcl authored May 26, 2025
  
  7a088067
- modify schedule_chunk_1f1b to fix the nccl communication bug · 6dcd0fb8
  dongcl authored May 26, 2025
  
  6dcd0fb8
20 May, 2025 1 commit
- support split_bw when te_version > 2.3.0.dev0 · 124accba
  dongcl authored May 20, 2025
  
  124accba
19 May, 2025 3 commits
- decorate _bwd_kernel_destindex_dequantize_kv with triton.jit · bfe0b4a9
  dongcl authored May 19, 2025
  
  bfe0b4a9
- megatron supports qcomm; 1f1b supports dense mlp · dc8a93ae
  dongcl authored May 19, 2025
  
  dc8a93ae
- patch for megatron core0.12.0 · fa142de0
  dongcl authored May 19, 2025
  
  fa142de0
18 May, 2025 2 commits
- switch Megatron to the git branch v0.12.0.rc3 · cf5d3189
  dongcl authored May 18, 2025
  
  cf5d3189
- patch for megatron v0.12.0.rc3. When tp > 1 and combined_1f1b = true, megatron... · b7c994f5
  dongcl authored May 18, 2025
```
patch for megatron v0.12.0.rc3. When tp > 1 and combined_1f1b = true, megatron (main) cannot execute properly
```
  b7c994f5
17 May, 2025 2 commits
- rewrite schedules based on megatron a73b4d2d4a993e9bea97fdebb841a393eb4ad5e7 · 08efd4ec
  dongcl authored May 17, 2025
  
  08efd4ec
- only transformer-engine>=2.4.0.dev supports split_bw · c964fcca
  dongcl authored May 17, 2025
  
  c964fcca
16 May, 2025 1 commit
- modify MoEAlltoAllPerBatchState, add tokens_per_expert attr · ab2a8334
  dongcl authored May 16, 2025
  
  ab2a8334
14 May, 2025 1 commit
- fix import error · 5890bb4c
  dongcl authored May 14, 2025
  
  5890bb4c
12 May, 2025 1 commit
- support a2a_overlap · 4bb958ec
  dongcl authored May 07, 2025
  
  4bb958ec
07 May, 2025 1 commit
- forward_backward_pipelining_without_interleaving supports a2a_overlap · 7c9dc3ec
  dongcl authored May 07, 2025
  
  7c9dc3ec
06 May, 2025 2 commits
- add schedules.py · 649bfbdb
  dongcl authored May 06, 2025
  
  649bfbdb
- a2a overlap · 32ee381a
  dongcl authored May 06, 2025
  
  32ee381a
30 Apr, 2025 1 commit
- support a2a overlap · 12b56c98
  dongcl authored Apr 30, 2025
  
  12b56c98
29 Apr, 2025 5 commits
- bug fix · 8551c38e
  silencealiang authored Apr 29, 2025
  
  8551c38e
- modify Megatron-LM · bcb9d73e
  dongcl authored Apr 29, 2025
  
  bcb9d73e
- modify megatron submodule · 02676184
  dongcl authored Apr 29, 2025
  
  02676184
- modify megatron submodule · 237dfffa
  dongcl authored Apr 29, 2025
  
  237dfffa
- patch for megatron 4429e8ebe · a8a2bbea
  dongcl authored Apr 29, 2025
  
  a8a2bbea
25 Apr, 2025 2 commits
- 删除llama2_7b.sh · 2ddbd4be
  dongcl authored Apr 25, 2025
  
  2ddbd4be
- bug fix · 8b656f45
  silencealiang authored Apr 25, 2025
  
  8b656f45
24 Apr, 2025 3 commits
- checkout Megatron-LM to main · 1f73f815
  silencealiang authored Apr 24, 2025
  
  1f73f815
- checkout Megatron-LM to main · c593bb40
  silencealiang authored Apr 24, 2025
  
  c593bb40
- merge megatron_v0.11.0 · 02eacdd3
  dongcl authored Apr 24, 2025
  
  02eacdd3