GitLab

feat: basic api key support

Feat: Clear cache during weight loading to prevent OOM on GPUs with <=8GB VRAM

feat:implementation of chat routing for Ollama

[UPDATE] Update documents.

Update DeepseekR1_V3_tutorial.md

Update DeepSeek-V3-Chat-multi-gpu-marlin.yaml

fix numa cpu distribution

fix dockerfile in devcontainer and fix expert torch

The numa node location would be calculated based on the total number
of worker threads.
So we should always use the actual number of threads instead of using a min() op.

📝 update benchmark.md

[update] Update doc.

⚡ release v0.2.2rc1

[release]  Release 0.2.2rc.

[update] Update readme.

 Update README.md

📝 add benchmark.md

⚡ update git ignore add docker dev container

Feat absorb for long prefill