Skip to content

GitLab

"src/libtorio/ffmpeg/pybind/pybind.cpp" did not exist on "c3310018fc936327e718b1341f6898bc1e08c6c2"
Unverified Commit 14efb155 authored by Atream's avatar Atream Committed by GitHub
Browse files

Update AMX.md

parent 38333cf1
Hide whitespace changes
Inline Side-by-side
Showing with 2 additions and 2 deletions
doc/en/AMX.md
View file @ 14efb155
...@@ -20,9 +20,9 @@ Here is the Qwen3MoE startup command: ...@@ -20,9 +20,9 @@ Here is the Qwen3MoE startup command:
``` python ``` python
# llamafile backend # llamafile backend
python ktransformers/server/main.py --architectures Qwen3MoeForCausalLM --model_path <model_dir> --gguf_path <gguf_dir> --optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml python ktransformers/server/main.py --architectures Qwen3MoeForCausalLM --model_path <model_dir> --gguf_path <gguf_dir> --optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve.yaml --backend_type balance_serve
# AMX backend # AMX backend
python ktransformers/server/main.py --architectures Qwen3MoeForCausalLM --model_path <model_dir> --gguf_path <gguf_dir> --optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve-amx.yaml python ktransformers/server/main.py --architectures Qwen3MoeForCausalLM --model_path <model_dir> --gguf_path <gguf_dir> --optimize_config_path ktransformers/optimize/optimize_rules/Qwen3Moe-serve-amx.yaml --backend_type balance_serve
``` ```
**Note: At present, Qwen3MoE running with AMX can only read BF16 GGUF; support for loading from safetensor will be added later.** **Note: At present, Qwen3MoE running with AMX can only read BF16 GGUF; support for loading from safetensor will be added later.**
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment