Commits · 1d497357d18e96215f3cf57ff5680aa5318084ec · evt_fugx1 / dcu_megatron

19 Jun, 2025 1 commit
- split routed experts and shared experts · 1d497357
  dongcl authored Jun 19, 2025
  
  1d497357
18 Jun, 2025 1 commit
- fix bug caused by mtp > 1 when using moe a2a overlap · 2ceeaafd
  dongcl authored Jun 18, 2025
  
  2ceeaafd
10 Jun, 2025 2 commits
- moe a2a overlap support self-attention; fix bug when use_shared_expert is false · 040838a0
  dongcl authored Jun 10, 2025
  
  040838a0
- fix parameters bug · 2a0c4358
  silencealiang authored Jun 10, 2025
  
  2a0c4358
09 Jun, 2025 1 commit
- add description for quantized communication · 3b081313
  dongcl authored Jun 09, 2025
  
  3b081313
03 Jun, 2025 1 commit
- update readme; add description for moe a2a overlap · 9b681ba5
  dongcl authored Jun 03, 2025
  
  9b681ba5
29 May, 2025 1 commit
- Merge branch 'a2a_overlap', support 1f1b overlap · 56819e16
  dongcl authored May 29, 2025
  
  56819e16
23 May, 2025 4 commits
- 添加llama和qwen的脚本 · 1e8185f4
  wangxj authored May 23, 2025
  
  1e8185f4
- Update README.md · 3e2c4b1e
  wxj authored May 23, 2025
  
  3e2c4b1e
- Update README.md · 3e3515a7
  wxj authored May 23, 2025
  
  3e3515a7
- update ParallelAttention to core v0.12.0 · 9be97c01
  dongcl authored May 23, 2025
  
  9be97c01
22 May, 2025 1 commit
- Update requirements.txt · 698bfd4d
  wxj authored May 22, 2025
  
  698bfd4d
21 May, 2025 1 commit
- update nccl and launch with binding · 30b15cb1
  silencealiang authored May 21, 2025
  
  30b15cb1
20 May, 2025 1 commit
- Update launch_with_binding.sh · cffad2c1
  silencealiang authored May 20, 2025
  
  cffad2c1
19 May, 2025 9 commits
- add launch_with_binding · e3cce568
  silencealiang authored May 19, 2025
  
  e3cce568
- support 1f1b overlap · 48b2dafd
  dongcl authored May 19, 2025
  
  48b2dafd
- Merge branch 'a2a_overlap' into 'core_v0.12.0' · c63fceee
  dongcl authored May 19, 2025
```
A2a overlap

See merge request OpenDAS/dcu_megatron!4
```
  c63fceee
- decorate _bwd_kernel_destindex_dequantize_kv with triton.jit · bfe0b4a9
  dongcl authored May 19, 2025
  
  bfe0b4a9
- megatron supports qcomm; 1f1b supports dense mlp · dc8a93ae
  dongcl authored May 19, 2025
  
  dc8a93ae
- update model parameters format · 6c3cfb1d
  silencealiang authored May 19, 2025
  
  6c3cfb1d
- switch Megatron-LM to git branch core_v0.12.0 · 935bfd74
  dongcl authored May 19, 2025
  
  935bfd74
- patch for megatron core_v0.12.0 · aa2c6708
  dongcl authored May 19, 2025
  
  aa2c6708
- patch for megatron core0.12.0 · fa142de0
  dongcl authored May 19, 2025
  
  fa142de0
18 May, 2025 2 commits
- switch Megatron to the git branch v0.12.0.rc3 · cf5d3189
  dongcl authored May 18, 2025
  
  cf5d3189
- patch for megatron v0.12.0.rc3. When tp > 1 and combined_1f1b = true, megatron... · b7c994f5
  dongcl authored May 18, 2025
```
patch for megatron v0.12.0.rc3. When tp > 1 and combined_1f1b = true, megatron (main) cannot execute properly
```
  b7c994f5
17 May, 2025 2 commits
- rewrite schedules based on megatron a73b4d2d4a993e9bea97fdebb841a393eb4ad5e7 · 08efd4ec
  dongcl authored May 17, 2025
  
  08efd4ec
- only transformer-engine>=2.4.0.dev supports split_bw · c964fcca
  dongcl authored May 17, 2025
  
  c964fcca
16 May, 2025 1 commit
- modify MoEAlltoAllPerBatchState, add tokens_per_expert attr · ab2a8334
  dongcl authored May 16, 2025
  
  ab2a8334
14 May, 2025 1 commit
- fix import error · 5890bb4c
  dongcl authored May 14, 2025
  
  5890bb4c
12 May, 2025 1 commit
- support a2a_overlap · 4bb958ec
  dongcl authored May 07, 2025
  
  4bb958ec
07 May, 2025 1 commit
- forward_backward_pipelining_without_interleaving supports a2a_overlap · 7c9dc3ec
  dongcl authored May 07, 2025
  
  7c9dc3ec
06 May, 2025 2 commits
- add schedules.py · 649bfbdb
  dongcl authored May 06, 2025
  
  649bfbdb
- a2a overlap · 32ee381a
  dongcl authored May 06, 2025
  
  32ee381a
30 Apr, 2025 1 commit
- support a2a overlap · 12b56c98
  dongcl authored Apr 30, 2025
  
  12b56c98
29 Apr, 2025 5 commits
- bug fix · 8551c38e
  silencealiang authored Apr 29, 2025
  
  8551c38e
- modify Megatron-LM · bcb9d73e
  dongcl authored Apr 29, 2025
  
  bcb9d73e
- modify megatron submodule · 02676184
  dongcl authored Apr 29, 2025
  
  02676184
- modify megatron submodule · 237dfffa
  dongcl authored Apr 29, 2025
  
  237dfffa
- patch for megatron 4429e8ebe · a8a2bbea
  dongcl authored Apr 29, 2025
  
  a8a2bbea
25 Apr, 2025 1 commit
- 删除llama2_7b.sh · 2ddbd4be
  dongcl authored Apr 25, 2025
  
  2ddbd4be