Skip to content

GitLab

Unverified Commit 76fa2d15 authored by Baizhou Zhang's avatar Baizhou Zhang Committed by GitHub
Browse files

Fix lora flashinfer import bug on ROCM (#3312)

parent 7ab84948
Hide whitespace changes
Inline Side-by-side
Showing with 4 additions and 1 deletion
python/sglang/srt/lora/backend/flashinfer_backend.py
View file @ 76fa2d15
from typing import Tuple from typing import Tuple
import torch import torch
from flashinfer import SegmentGEMMWrapper
from sglang.srt.lora.backend import BaseLoraBackend from sglang.srt.lora.backend import BaseLoraBackend
from sglang.srt.lora.lora import LoraBatchInfo from sglang.srt.lora.lora import LoraBatchInfo
from sglang.srt.utils import is_flashinfer_available
if is_flashinfer_available():
from flashinfer import SegmentGEMMWrapper
class FlashInferLoraBackend(BaseLoraBackend): class FlashInferLoraBackend(BaseLoraBackend):
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment