Unverified Commit 3a100b92 authored by Varun Sundar Rabindranath's avatar Varun Sundar Rabindranath Committed by GitHub
Browse files

[Bugfix] LoRA : Fix the order in which the kernels process LoRAs (#16040)


Signed-off-by: default avatarVarun Sundar Rabindranath <varun@neuralmagic.com>
Co-authored-by: default avatarVarun Sundar Rabindranath <varun@neuralmagic.com>
parent 242a637a
...@@ -111,7 +111,7 @@ class LoRAKernelMeta: ...@@ -111,7 +111,7 @@ class LoRAKernelMeta:
# active_lora_ids, num_tokens_per_lora # active_lora_ids, num_tokens_per_lora
lora_ids, num_tokens_per_lora = torch.unique(token_lora_mapping, lora_ids, num_tokens_per_lora = torch.unique(token_lora_mapping,
sorted=False, sorted=True,
return_counts=True) return_counts=True)
self.active_lora_ids[:lora_ids.size(0)].copy_(lora_ids, self.active_lora_ids[:lora_ids.size(0)].copy_(lora_ids,
non_blocking=True) non_blocking=True)
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment