server/text_generation_server/models/flash_causal_lm.py · 5f32dea1e2e6f1b4a86773ef7f1e8861fca5c61d · OpenDAS / text-generation-inference

fix: prefer inplace softmax to avoid copy (#2661) · 5f32dea1

drbh authored Oct 17, 2024



* fix: prefer inplace softmax to avoid copy

* Update server/text_generation_server/models/flash_causal_lm.py
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>

---------
Co-authored-by: Nicolas Patry <patry.nicolas@protonmail.com>

5f32dea1

flash_causal_lm.py 88.9 KB

Replace flash_causal_lm.py