• fxmarty's avatar
    Add RoCm support (#1243) · b2b5df0e
    fxmarty authored
    
    
    This PR adds support for AMD Instinct MI210 & MI250 GPUs, with paged
    attention and FAv2 support.
    
    Remaining items to discuss, on top of possible others:
    * Should we have a
    `ghcr.io/huggingface/text-generation-inference:1.1.0+rocm` hosted image,
    or is it too early?
    * Should we set up a CI on MI210/MI250? I don't have access to the
    runners of TGI though.
    * Are we comfortable with those changes being directly in TGI, or do we
    need a fork?
    
    ---------
    Co-authored-by: default avatarFelix Marty <felix@hf.co>
    Co-authored-by: default avatarOlivierDehaene <olivier@huggingface.co>
    Co-authored-by: default avatarYour Name <you@example.com>
    b2b5df0e
build.yaml 11.9 KB