GitLab

Support for IQ1_S(Dynamic 1.58-bit)

fix: fix SSE formatting

feat: basic api key support

Feat: Clear cache during weight loading to prevent OOM on GPUs with <=8GB VRAM

feat:implementation of chat routing for Ollama

[UPDATE] Update documents.

Update DeepseekR1_V3_tutorial.md

Update DeepSeek-V3-Chat-multi-gpu-marlin.yaml

fix numa cpu distribution

fix dockerfile in devcontainer and fix expert torch

The numa node location would be calculated based on the total number
of worker threads.
So we should always use the actual number of threads instead of using a min() op.

📝 update benchmark.md

[update] Update doc.

⚡ release v0.2.2rc1

[release]  Release 0.2.2rc.

[update] Update readme.

 Update README.md