Commits · cecc9b5db6867c1019df3ba3af85a157d45dc77e · jerrrrry / infinilm

16 Jul, 2025 1 commit
- support iluvatar device type · cecc9b5d
  zhangyue authored Jul 16, 2025
  
  cecc9b5d
10 Jul, 2025 2 commits
- fix: assign scale_* as class members instead of local variables · 1549eced
  mxCynic authored Jul 10, 2025
  
  1549eced
- fix: Use model_type to determine whether to load scale related parameters · 45eecde6
  mxCynic authored Jul 10, 2025
  
  45eecde6
09 Jul, 2025 2 commits
- fix: fix 9G4B model by add coefficent to weight tensor when load · 0fca9576
  mxCynic authored Jul 09, 2025
  
  0fca9576
- support bf16 · a5deda33
  PanZezhong authored Jul 09, 2025
  
  a5deda33
07 Jul, 2025 1 commit
- issue/14: 修复非流式输出bug · 4fb73db2
  Catheriany authored Jul 07, 2025
  
  4fb73db2
04 Jul, 2025 1 commit
- issue/14: 兼容openai协议 · 9f5e40d6
  Catheriany authored Jul 04, 2025
  
  9f5e40d6
27 Jun, 2025 2 commits
- support max-token and max-batch args · 7a087fdc
  PanZezhong authored Jun 27, 2025
  
  7a087fdc
- 9G7B support safetensors · 732c7f04
  Pan Zezhong authored Jun 27, 2025
  
  732c7f04
25 Jun, 2025 2 commits
- refactor task to avoid model dependency on async libs · 2b9ce5a6
  Pan Zezhong authored Jun 25, 2025
  
  2b9ce5a6
- handle client abort · cfc8b598
  Pan Zezhong authored Jun 25, 2025
  
  cfc8b598
24 Jun, 2025 1 commit
- Support continous batching · dcd6693f
  Pan Zezhong authored Jun 24, 2025
  
  dcd6693f
23 Jun, 2025 2 commits
- add kv cache pool · 21f83e91
  Pan Zezhong authored Jun 23, 2025
  
  21f83e91
- change url · a73433ab
  Pan Zezhong authored Jun 23, 2025
  
  a73433ab
20 Jun, 2025 1 commit
- add lock for single request handling, add test server script · 53cb1582
  Pan Zezhong authored Jun 20, 2025
  
  53cb1582
18 Jun, 2025 1 commit
- support transposed weights config · 392f0c26
  PanZezhong authored Jun 18, 2025
  
  392f0c26
04 Jun, 2025 1 commit
- add server script · 51b1aade
  Pan Zezhong authored Jun 04, 2025
  
  51b1aade
27 May, 2025 1 commit
- support qwen2 · b0010cbc
  PanZezhong authored May 27, 2025
  
  b0010cbc
26 May, 2025 1 commit
- fix workspace · 13f98ed3
  PanZezhong authored May 26, 2025
  
  13f98ed3
22 May, 2025 1 commit
- support 9g7b · 366d3aef
  Pan Zezhong authored May 22, 2025
  
  366d3aef
20 May, 2025 1 commit
- Add workspace allocator · 5540d53a
  PanZezhong authored May 21, 2025
  
  5540d53a
18 May, 2025 1 commit
- llama cpu · 967bcb64
  PanZezhong authored May 19, 2025
  
  967bcb64
15 May, 2025 2 commits
- fix xmake · 760b769e
  PanZezhong authored May 15, 2025
  
  760b769e
- add infer script · e435558f
  PanZezhong authored May 15, 2025
  
  e435558f
14 May, 2025 1 commit
- init · 81fe2ba3
  PanZezhong authored May 14, 2025
  
  81fe2ba3