src/transformers/models/llama/modeling_llama.py · cc4a664baaac790aadc4ca9c5d93031893432433 · chenpangpang / transformers

`torch.compile` compatibility with `generate` + static cache (#29114) · cc4a664b

fxmarty authored Feb 21, 2024



* fix compatibility

* working version

* cleanup

* sanity checks

* more sanity

* working version WITH refactor

* working without API change

* cleanup & tests pass

* more cleaning

* fix test

* fix tests

* Update src/transformers/generation/utils.py
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

* smaller comment

* update comment

* update comment

---------
Co-authored-by: Arthur <48595927+ArthurZucker@users.noreply.github.com>

cc4a664b

modeling_llama.py 68.9 KB

Replace modeling_llama.py