• rocking's avatar
    layernorm and groupnorm backward data (#1083) · a69aa2a1
    rocking authored
    * rename folder
    
    * Add type string
    
    * Remove typo
    
    * Add deviceOp to backward x
    
    * Add comment to describe the behavior of backward normalization
    
    * Add kernel function, prepare to implement
    
    * implement generic kernel
    
    * Check vector size
    
    * Add sweep once pipeline for small reduce size
    
    * Fix bug of KRaw_ error
    
    * Fix bug of dx stride
    
    * sanity check for mean and rstd
    
    * backward x for groupnorm
    
    * Add bwd x instance
    
    * add layernorm 2d bwd gamma beta instances
    
    * Change save mean var type from f32 to f16 in f16 mode
    
    * Change the example to f16
    
    * Add groupnorm bwd gamma beta instance
    
    * Add groupnorm bwd x instance
    
    * Fix naming
    
    * Add layernorm bwd x ckprofiler
    
    * Add groupnorm bwd x profiler
    
    * clang format
    
    * Rename bwd x to bwd data
    
    * Fix bug of verification in profiler
    
    * Add test of layernorm and groupnorm bwd data
    
    * Add missing cmake
    
    * Add layernorm2d bwd data
    
    * rename fwd example
    
    * Add groupnorm client example
    
    * Fix typo. replace Invarient with Invariant
    
    * Add checking before running the best instance
    a69aa2a1
softmax4d.cpp 6.51 KB