1. 25 Mar, 2021 1 commit
    • Paul Fultz II's avatar
      Add cpu fusion for gelu and layernorm (#761) · 728d083d
      Paul Fultz II authored
      
      
      * Add eliminate_data_type pass
      
      * Formatting
      
      * Auto convert quant ops
      
      * Formatting
      
      * Flip the order of decompose
      
      * Compute max size differently
      
      * Formatting
      
      * Clamp values in convert
      
      * Formatting
      
      * Fix loss of precision in reduce
      
      * Formatting
      
      * Fix bugs in reduction
      
      * Fix accumulator type in reference softmax implementation
      
      * Formatting
      
      * Update convert test
      
      * Remove unused variables
      
      * Remove unnecessary quant_dot check
      
      * Formatting
      
      * Add tests
      
      * Formatting
      
      * Remove unused code
      
      * Remove duplicate ops
      
      * Remove blaze dependency
      
      * Use set since shape::type_t is no hashable on gcc 5
      
      * Formatting
      
      * Add dnnl binary op
      
      * Formatting
      
      * Add binary and eltwise
      
      * Formatting
      
      * Add softmax
      
      * Formatting
      
      * Remove unused operators
      
      * Add missing files
      
      * Formatting
      
      * Add lrn
      
      * Formatting
      
      * Add deconvolution
      
      * Formatting
      
      * Change allocate default
      
      * Add reorder
      
      * Formatting
      
      * Add reductions
      
      * Formatting
      
      * Sort lines
      
      * Change literals in another loop
      
      * Add pow operator
      
      * Formatting
      
      * Add pow operator
      
      * Formatting
      
      * Make sure shapes are packed
      
      * Allow broadcasted inputs
      
      * Remove unused operators
      
      * Simplify functions
      
      * Remove softmax
      
      * Add sub and erf functions
      
      * Formatting
      
      * Fix bug
      
      * Formatting
      
      * Improve parallism
      
      * Formatting
      
      * Allow multiple batch dimensions
      
      * Formatting
      
      * Move literal transforms out of lowering
      
      * Formatting
      
      * Add gather operator
      
      * Sort lines
      
      * Add early exit for carry
      
      * Formatting
      
      * Add missing concat
      
      * Rename macro
      
      * Fix deep nesting
      
      * Formatting
      
      * Fix cppcheck issues
      
      * Remov else
      
      * Move attribute to typedef
      
      * Formatting
      
      * Disable maybe-uninitialized warning since its broken on gcc
      
      * Add constexpr default constructor
      
      * Formatting
      
      * Fix compiler warnings
      
      * Fix adjust_allocation test
      
      * Add layernorm matcher
      
      * Add gelu_erf matcher
      
      * Formatting
      
      * Add gelu_tanh matcher
      
      * Formatting
      
      * Remove match namespace
      
      * Formatting
      
      * Use matcher instead of string
      
      * Formatting
      
      * Add fusions
      
      * Formatting
      
      * Make input a const ref
      
      * Make this explicit for gcc 5
      Co-authored-by: default avatarShucai Xiao <shucai@gmail.com>
      Co-authored-by: default avatarmvermeulen <5479696+mvermeulen@users.noreply.github.com>
      728d083d