• AllentDan's avatar
    fix benchmark serving computation mistake (#630) · 529e56bd
    AllentDan authored
    * fix benchmark serving computation mistake
    
    * fix timestamps computations
    
    * remove speed up
    
    * no mp
    
    * mp seems faster?
    
    * remove
    
    * update
    
    * remove
    
    * fix
    
    * update
    
    * update print log
    
    * typo
    
    * print fist token latency only stream==True
    
    * remove renew_session
    
    * update AsyncEngine
    529e56bd
profile_serving.py 7.23 KB