- 30 Sep, 2024 1 commit
-
-
Daniël de Kok authored
This change uses the updated Marlin MoE kernel from vLLM to support MoE with activation sorting and groups.
-
- 31 Jul, 2024 1 commit
-
-
Daniël de Kok authored
The `GPTWeightLoader` was structured like this in pseudocode: if marlin: Set up tensors in a way that GPTQ-Marlin expects else: Set up tensors in a way that ExLlama/GPTQ/AWQ expect However, the GPT-Marlin implementation details should really be in the `marlin` module. So move the former part out to a separate `GPTQMarlinWeightsLoader`.
-
- 29 Jul, 2024 1 commit
-
-
Daniël de Kok authored
-
- 24 Jul, 2024 1 commit
-
-
Daniël de Kok authored
The marlin.py file was getting large, split it up.
-