Commits · 8f1dc2a482674c53f1fb3ecae87f78cb9a9fd3df · jerrrrry / infinilm

16 Mar, 2026 1 commit
- issue/265 perf(llm): replace O(n²) full-sequence detokenize with incremental decode · 8f1dc2a4
  MaYuhang authored Mar 16, 2026
  
  8f1dc2a4
11 Mar, 2026 4 commits
- Merge pull request #249 from InfiniTensor/Issue/243 · 5fb56f97
  thatPepe authored Mar 11, 2026
```
Issue/243:支持w4a16 awq fp16推理
```
  5fb56f97
- Fix: Add lifecycle management to AWQ linear function · fc97bbd8
  qinyiqun authored Mar 11, 2026
  
  fc97bbd8
- add mha_kvcache (#261) · a256e8d9
  suss authored Mar 11, 2026
```
* add mha_kvcache

* repair gqa-api bug
```
  a256e8d9
- Merge pull request #240 from InfiniTensor/issue/239 · 6ab9ee22
  thatPepe authored Mar 11, 2026
```
issue/239 bench vllm script
```
  6ab9ee22
10 Mar, 2026 1 commit
- Merge pull request #262 from InfiniTensor/issue/244 · 22711ec5
  thatPepe authored Mar 10, 2026
```
issue/244 feat(llm): add prefix cache reuse for static KV cache
```
  22711ec5
09 Mar, 2026 5 commits
- issue/244 feat(llm): add prefix cache reuse for static KV cache · a89b194a
  MaYuhang authored Mar 09, 2026
  
  a89b194a
- Merge pull request #260 from InfiniTensor/issue/259 · 3b8e1cb7
  thatPepe authored Mar 09, 2026
```
issue/259 - add attn backend option to inference server
```
  3b8e1cb7
- issue/259 - add attn backend option to inference server · 91cd2992
  wooway777 authored Mar 09, 2026
  
  91cd2992
- Merge pull request #258 from InfiniTensor/issue/257 · dfec9d89
  thatPepe authored Mar 09, 2026
```
issue/257 fix(llm): sync inference service with FLA engine interface …
```
  dfec9d89
- issue/257 fix(llm): sync inference service with FLA engine interface changes · 30e93254
  MaYuhang authored Mar 09, 2026
  
  30e93254
06 Mar, 2026 5 commits
- Merge pull request #234 from InfiniTensor/issue/233 · 70561bd9
  thatPepe authored Mar 06, 2026
```
issue/233 fix: improve request lifecycle management and timeout handling
```
  70561bd9
- Merge pull request #250 from InfiniTensor/issue/248 · d09de04c
  thatPepe authored Mar 06, 2026
```
Issue/248 support flash-attention
```
  d09de04c
- issue/248 - change default attn backend to classic impl · 5dc85bf4
  wooway777 authored Mar 06, 2026
  
  5dc85bf4
- issue/248 - replace __C with __INFINI_C · 84fbe5b1
  wooway777 authored Mar 05, 2026
  
  84fbe5b1
- issue/248 fix reset cache · fcbf7bfc
  PanZezhong authored Mar 06, 2026
  
  fcbf7bfc
05 Mar, 2026 6 commits
- issue/248 - support attn backend in front end and update readme · 471309e2
  wooway777 authored Mar 05, 2026
  
  471309e2
- issue/248 fix total seqlen to cpu as int32 · 0ea1cd55
  PanZezhong authored Mar 05, 2026
  
  0ea1cd55
- issue/248 optimize: use flash-attn only in prefill · 8297a0b7
  PanZezhong authored Mar 04, 2026
  
  8297a0b7
- issue/248 add arg for flash-attn backend · ae210024
  PanZezhong authored Mar 03, 2026
  
  ae210024
- issue/248 support flash-attention lib · 7668db4f
  PanZezhong authored Mar 03, 2026
  
  7668db4f
- Merge pull request #247 from InfiniTensor/issue/246 · f67956fe
  PanZezhong1725 authored Mar 05, 2026
```
issue/246 change default kvcache blocksize to 256
```
  f67956fe
04 Mar, 2026 1 commit
- Merge pull request #252 from InfiniTensor/issue/251 · 08090824
  thatPepe authored Mar 04, 2026
```
issue/251 - change nt config to op config for kv caching
```
  08090824
03 Mar, 2026 1 commit
- issue/251 - change nt config to op config for kv caching · 2ae79b92
  wooway777 authored Mar 03, 2026
  
  2ae79b92
02 Mar, 2026 2 commits
- Issue/243:支持w4a16 awq fp16推理 · f692d681
  qinyiqun authored Feb 27, 2026
  
  f692d681
- Merge pull request #242 from InfiniTensor/issue/241 · 6ae48322
  gongchensu authored Mar 02, 2026
```
issue/241 fix mmlu test, add vllm support
```
  6ae48322
27 Feb, 2026 1 commit
- issue/239 adjust benchmark · e6df4acf
  PanZezhong authored Feb 27, 2026
  
  e6df4acf
26 Feb, 2026 1 commit
- issue/239 bench vllm script · e0e5827f
  PanZezhong authored Feb 26, 2026
  
  e0e5827f
25 Feb, 2026 1 commit
- issue/241 fix mmlu test, add vllm support · 0086ff2f
  PanZezhong authored Feb 25, 2026
  
  0086ff2f
24 Feb, 2026 2 commits
- Merge pull request #238 from InfiniTensor/issue/237 · e76bb324
  thatPepe authored Feb 24, 2026
```
issue/237 - support hygon in bench and inf server
```
  e76bb324
- issue/237 - support hygon in bench and inf server · 8f71a5ec
  wooway777 authored Feb 24, 2026
  
  8f71a5ec
13 Feb, 2026 6 commits
- Merge pull request #236 from InfiniTensor/issue/235 · 39b594ff
  thatPepe authored Feb 13, 2026
```
issue/235 - expand bench prompt
```
  39b594ff
- issue/235 - expand bench prompt · 26b6b92e
  wooway777 authored Feb 13, 2026
  
  26b6b92e
- issue/233 fix: improve request lifecycle management and timeout handling · d45ad1e6
  MaYuhang authored Feb 13, 2026
  
  d45ad1e6
- Merge pull request #232 from InfiniTensor/issue/231 · 323c78a1
  thatPepe authored Feb 13, 2026
```
issue/231 - remove ninetoothed dependency by default
```
  323c78a1
- issue/231 - remove ninetoothed dependency by default · 06be2dc3
  wooway777 authored Feb 13, 2026
  
  06be2dc3
- Merge pull request #205 from InfiniTensor/demo131 · a4ced800
  thatPepe authored Feb 13, 2026
```
Demo-131 Cuda graph with optimized paged attention
```
  a4ced800
12 Feb, 2026 3 commits
- issue/229 - fix: fix inference_server.py for moore gpu · 04c37f3f
  zhushuang authored Feb 12, 2026
  
  04c37f3f
- issue/221 - patch encode plus for ali · f053fe1d
  wooway777 authored Feb 12, 2026
  
  f053fe1d
- Merge pull request #227 from InfiniTensor/issue/226 · 0879747e
  thatPepe authored Feb 12, 2026
```
Issue/226：add warmup before InfiniLM bench.py generation
```
  0879747e