• myhloli's avatar
    perf(inference): adjust batch ratio thresholds for GPU memory sizes · 74e954da
    myhloli authored
    - Increase batch ratio to 32 for GPU memory >= 24GB
    - Set batch ratio to 16 for GPU memory >= 16GB
    - Reduce batch ratio to 8 for GPU memory >= 12GB
    - Lower batch ratio to 4 for GPU memory >= 8GB
    - Set batch ratio to 2 for GPU memory >= 6GB
    - Keep batch ratio at 1 for lower GPU memory sizes
    74e954da
doc_analyze_by_custom_model.py 11.6 KB