• Thor Johnsen's avatar
    Multi tensor lamb optimizer (#334) · 8be5b6be
    Thor Johnsen authored
    * First draft, for discussion
    
    * Fix mistakes in LAMB equations
    
    * Add loop over chunk
    
    * Bug fix
    
    * Bug fix
    
    * Bug fix
    
    * Undo bug fix
    
    * Bug fix
    
    * Add multi tensor LAMB optimizer to setup.py
    
    * Rename step_size to learning_rate
    
    * Fix compilation errors
    8be5b6be
amp_C_frontend.cpp 1.73 KB