v1.0

e4575be9 · huaerkl · e4575be9 · e4575be9 · e4575be9 · e4575be9
Commit e4575be9 authored Aug 04, 2023 by huaerkl
20 changed files
--- a/megatron/fp16_deprecated/loss_scaler.py
+++ b/megatron/fp16_deprecated/loss_scaler.py
--- a/megatron/fused_kernels/__init__.py
+++ b/megatron/fused_kernels/__init__.py
--- a/megatron/fused_kernels/compat.h
+++ b/megatron/fused_kernels/compat.h
--- a/megatron/fused_kernels/layer_norm_cuda.cpp
+++ b/megatron/fused_kernels/layer_norm_cuda.cpp
--- a/megatron/fused_kernels/layer_norm_cuda_kernel.cu
+++ b/megatron/fused_kernels/layer_norm_cuda_kernel.cu
--- a/megatron/fused_kernels/layer_norm_hip_kernel.hip
+++ b/megatron/fused_kernels/layer_norm_hip_kernel.hip
--- a/megatron/fused_kernels/scaled_masked_softmax.cpp
+++ b/megatron/fused_kernels/scaled_masked_softmax.cpp
--- a/megatron/fused_kernels/scaled_masked_softmax.h
+++ b/megatron/fused_kernels/scaled_masked_softmax.h
--- a/megatron/fused_kernels/scaled_masked_softmax_cuda.cu
+++ b/megatron/fused_kernels/scaled_masked_softmax_cuda.cu
--- a/megatron/fused_kernels/scaled_masked_softmax_hip.hip
+++ b/megatron/fused_kernels/scaled_masked_softmax_hip.hip
--- a/megatron/fused_kernels/scaled_upper_triang_masked_softmax.cpp
+++ b/megatron/fused_kernels/scaled_upper_triang_masked_softmax.cpp
--- a/megatron/fused_kernels/scaled_upper_triang_masked_softmax.h
+++ b/megatron/fused_kernels/scaled_upper_triang_masked_softmax.h
--- a/megatron/fused_kernels/scaled_upper_triang_masked_softmax_cuda.cu
+++ b/megatron/fused_kernels/scaled_upper_triang_masked_softmax_cuda.cu
--- a/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.hip
+++ b/megatron/fused_kernels/scaled_upper_triang_masked_softmax_hip.hip
--- a/megatron/fused_kernels/tests/__init__.py
+++ b/megatron/fused_kernels/tests/__init__.py
--- a/megatron/fused_kernels/tests/test_fused_kernels.py
+++ b/megatron/fused_kernels/tests/test_fused_kernels.py
--- a/megatron/fused_kernels/type_shim.h
+++ b/megatron/fused_kernels/type_shim.h
--- a/megatron/global_vars.py
+++ b/megatron/global_vars.py
--- a/megatron/indexer.py
+++ b/megatron/indexer.py
--- a/megatron/initialize.py
+++ b/megatron/initialize.py