Commits · 8110e44529f431d54b02060528601c0d3e3f7d02 · OpenDAS / vllm_cscc · GitLab

17 Sep, 2024 15 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [CI/Build] Excluding kernels/test_gguf.py from ROCm (#8520) · 09deb472
  Alexey Kondratiev(AMD) authored Sep 17, 2024
  
  09deb472
- [doc] improve installation doc (#8550) · fa0c114f
  youkaichao authored Sep 17, 2024
```
Co-authored-by: Andy Dai <76841985+Imss27@users.noreply.github.com>
```
  fa0c114f
- [Bugfix] Fix TP > 1 for new granite (#8544) · 98f97133
  Joe Runde authored Sep 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  98f97133
- [Misc] Don't dump contents of kvcache tensors on errors (#8527) · 56c3de01
  Nick Hill authored Sep 17, 2024
  
  56c3de01
- [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) · a54ed802
  Patrick von Platen authored Sep 17, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a54ed802
- [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) · 9855b995
  chenqianfzh authored Sep 17, 2024
  
  9855b995
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
- [Benchmark] Support sample from HF datasets and image input for benchmark_serving (#8495) · 1b6de835
  Isotr0py authored Sep 17, 2024
  
  1b6de835
- [Misc] Limit to ray[adag] 2.35 to avoid backward incompatible change (#8509) · cbdb2522
  Rui Qiao authored Sep 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  cbdb2522
- [torch.compile] register allreduce operations as custom ops (#8526) · 99aa4edd
  youkaichao authored Sep 16, 2024
  
  99aa4edd
- [Misc][Bugfix] Disable guided decoding for mistral tokenizer (#8521) · ee2bceaa
  Roger Wang authored Sep 16, 2024
  
  ee2bceaa
- [Frontend] Improve Nullable kv Arg Parsing (#8525) · 1c1bb388
  Alex Brooks authored Sep 16, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  1c1bb388
- [refactor] remove triton based sampler (#8524) · 546034b4
  Simon Mo authored Sep 16, 2024
  
  546034b4
- [Bugfix] Fix 3.12 builds on main (#8510) · cca61642
  Joe Runde authored Sep 16, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cca61642
16 Sep, 2024 9 commits
- [misc] small qol fixes for release process (#8517) · 5ce45eb5
  Simon Mo authored Sep 16, 2024
  
  5ce45eb5
- [perf bench] set timeout to debug hanging (#8516) · 5478c4b4
  Simon Mo authored Sep 16, 2024
  
  5478c4b4
- [Bugfix] Bind api server port before starting engine (#8491) · 47f5e03b
  Kevin Lin authored Sep 16, 2024
  
  47f5e03b
- [doc] update doc on testing and debugging (#8514) · 2759a43a
  youkaichao authored Sep 16, 2024
  
  2759a43a
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [Bugfix][Kernel] Fix build for sm_60 in GGUF kernel (#8506) · 781e3b9a
  sasha0552 authored Sep 16, 2024
  
  781e3b9a
- [BugFix] Fix clean shutdown issues (#8492) · acd5511b
  Nick Hill authored Sep 16, 2024
  
  acd5511b
- [Frontend] Expose revision arg in OpenAI server (#8501) · 837c1968
  lewtun authored Sep 16, 2024
  
  837c1968
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 2 commits
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
- [Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by updating type annotations (#8490) · 3724d5f6
  Chris authored Sep 15, 2024
  
  3724d5f6
14 Sep, 2024 5 commits
- [TPU] Implement multi-step scheduling (#8489) · 50e9ec41
  Woosuk Kwon authored Sep 14, 2024
  
  50e9ec41
- [torch.compile] add a flag to disable custom op (#8488) · 47790f3e
  youkaichao authored Sep 14, 2024
  
  47790f3e
- [torch.compile] fix functionalization (#8480) · a36e070d
  youkaichao authored Sep 14, 2024
  
  a36e070d
- [Model] support minicpm3 (#8297) · 8a0cf1dd
  ywfang authored Sep 14, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8a0cf1dd
- [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) · 1ef0d2ef
  Charlie Fu authored Sep 13, 2024
  
  1ef0d2ef
13 Sep, 2024 9 commits
- [Hardware][intel GPU] bump up ipex version to 2.3 (#8365) · 85172520
  Kunshang Ji authored Sep 14, 2024
```
Co-authored-by: Yan Ma <yan.ma@intel.com>
```
  85172520
- bump version to v0.6.1.post2 (#8473) · 9ba0817f
  Simon Mo authored Sep 13, 2024
  
  9ba0817f
- [HotFix] Fix final output truncation with stop string + streaming (#8468) · 18e9e1f7
  Nick Hill authored Sep 13, 2024
  
  18e9e1f7
- [Doc] Add oneDNN installation to CPU backend documentation (#8467) · f57092c0
  Isotr0py authored Sep 14, 2024
  
  f57092c0
- [CI/Build] Reorganize models tests (#7820) · a84e598e
  Cyrus Leung authored Sep 14, 2024
  
  a84e598e
- [plugin][torch.compile] allow to add custom compile backend (#8445) · 0a4806f0
  youkaichao authored Sep 13, 2024
  
  0a4806f0
- [Installation] Gate FastAPI version for Python 3.8 (#8456) · ecd7a1d5
  Cyrus Leung authored Sep 14, 2024
  
  ecd7a1d5
- [misc][ci] fix quant test (#8449) · a2469127
  youkaichao authored Sep 13, 2024
  
  a2469127
- [Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (#8442) · 06311e29
  Jee Jee Li authored Sep 13, 2024
  
  06311e29