Commits · 344c988d1dcea878e65eb8cfb637eae621fc65b7 · gaoqiong / flash-attention

21 Jan, 2024 1 commit
- return z_loss (#768) · d8aacc51
  Curtis "Fjord" Hawthorne authored Jan 21, 2024
  
  d8aacc51
17 Dec, 2023 2 commits
- [CrossEntropy] Test longer sequences · 713bd3aa
  Tri Dao authored Dec 16, 2023
  
  713bd3aa
- [CrossEntropy] Implement logit_scale option · 08124c8f
  Tri Dao authored Dec 16, 2023
  
  08124c8f
20 Nov, 2023 2 commits
- [CrossEntropy] Simplify the case of large vocab with Tensor Parallel · aaa14741
  Tri Dao authored Nov 19, 2023
  
  aaa14741
- fix flash ce mp large vocab (#673) · abf04a56
  Shijie authored Nov 20, 2023
  
  abf04a56
16 Sep, 2023 1 commit
- [CE] Implement CrossEntropyLoss in Triton · 5400fdc4
  Tri Dao authored Sep 15, 2023
  
  5400fdc4
19 Aug, 2023 1 commit
- Run isort and black on test files · 0e8c46ae
  Tri Dao authored Aug 18, 2023
  
  0e8c46ae
27 Dec, 2022 2 commits
- Tweak CrossEntropyLoss to take process_group in init · c6ecd40a
  Tri Dao authored Dec 27, 2022
  
  c6ecd40a
- Implement Tensor Parallel for GPT model · b4018a50
  Tri Dao authored Dec 25, 2022
  
  b4018a50
23 Dec, 2022 1 commit
- Add smoothing for CrossEntropyParallel, rename to CrossEntropyLoss · dff68c2b
  Tri Dao authored Dec 23, 2022
  
  dff68c2b
14 Nov, 2022 1 commit
- Make nccl operations async in CrossEntropyLossParallel · 343492ec
  Tri Dao authored Nov 13, 2022
  
  343492ec
13 Nov, 2022 1 commit
- Add fused cross entropy loss · 7c995381
  Tri Dao authored Nov 12, 2022
  
  7c995381