Commits · d23679eb9960ad2a876b88ebd0028dbe55c3172a · OpenDAS / vllm_cscc · GitLab

22 Sep, 2024 6 commits
- [SpecDec][Misc] Cleanup, remove bonus token logic. (#8701) · c6bd70d7
  Lily Liu authored Sep 22, 2024
  
  c6bd70d7
- [Model][VLM] Add LLaVA-Onevision model support (#8486) · 5b595327
  litianjian authored Sep 23, 2024
```
Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  5b595327
- [MISC] rename CudaMemoryProfiler to DeviceMemoryProfiler (#8703) · ca2b628b
  Huazhong Ji authored Sep 23, 2024
  
  ca2b628b
- [Model] Refactor BLIP/BLIP-2 to support composite model loading (#8407) · 06ed2815
  Cyrus Leung authored Sep 22, 2024
  
  06ed2815
- [ci][build] fix vllm-flash-attn (#8699) · 0e40ac9b
  youkaichao authored Sep 21, 2024
  
  0e40ac9b
- [Bugfix] Refactor composite weight loading logic (#8656) · 13d88d41
  Isotr0py authored Sep 22, 2024
  
  13d88d41
21 Sep, 2024 7 commits
- [dbrx] refactor dbrx experts to extend FusedMoe class (#8518) · 9dc7c6c7
  Divakar Verma authored Sep 21, 2024
  
  9dc7c6c7
- [Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x speedup... · ec4aaad8
  rasmith authored Sep 21, 2024
```
[Kernel][Triton][AMD] Remove tl.atomic_add from awq_gemm_kernel, 2-5x speedup MI300, minor improvement for MI250 (#8646)
```
  ec4aaad8
- [VLM] Use `SequenceData.from_token_counts` to create dummy data (#8687) · 5e85f4f8
  Cyrus Leung authored Sep 21, 2024
  
  5e85f4f8
- [Kernel] Build flash-attn from source (#8245) · 71c60491
  Luka Govedič authored Sep 21, 2024
  
  71c60491
- [Core] Factor out common code in `SequenceData` and `Sequence` (#8675) · 0455c46e
  Cyrus Leung authored Sep 21, 2024
  
  0455c46e
- [MISC] add support custom_op check (#8557) · d4bf085a
  Kunshang Ji authored Sep 21, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  d4bf085a
- [Core] Rename `PromptInputs` and `inputs`(#8673) · 0057894e
  Cyrus Leung authored Sep 21, 2024
  
  0057894e
20 Sep, 2024 8 commits
- [Bugfix] Fix incorrect llava next feature size calculation (#8496) · 0f961b3c
  zyddnys authored Sep 20, 2024
  
  0f961b3c
- [Bugfix][Core] Fix tekken edge case for mistral tokenizer (#8640) · b4e4eda9
  Patrick von Platen authored Sep 20, 2024
  
  b4e4eda9
- [Bugfix] Config got an unexpected keyword argument 'engine' (#8556) · 2874bac6
  Pastel！ authored Sep 21, 2024
  
  2874bac6
- [Bugfix] Validate SamplingParam n is an int (#8548) · b28298f2
  saumya-saran authored Sep 20, 2024
  
  b28298f2
- [Model] Add OLMoE (#7922) · 3b63de93
  Niklas Muennighoff authored Sep 20, 2024
  
  3b63de93
- [Core] Support Lora lineage and base model metadata management (#6315) · 260d40b5
  Jiaxin Shan authored Sep 19, 2024
  
  260d40b5
- [bugfix] [AMD] add multi-step advance_step to ROCmFlashAttentionMetadata (#8474) · 9e5ec35b
  William Lin authored Sep 19, 2024
  
  9e5ec35b
- [Bugfix] Fix Phi3.5 mini and MoE LoRA inference (#8571) · 18ae428a
  Amit Garg authored Sep 19, 2024
  
  18ae428a
19 Sep, 2024 5 commits
- [Core] simplify logits resort in _apply_top_k_top_p (#8619) · e42c634a
  盏一 authored Sep 20, 2024
  
  e42c634a
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
- [Frontend] Use MQLLMEngine for embeddings models too (#8584) · 76515f30
  Nick Hill authored Sep 19, 2024
  
  76515f30
- Revert "[Misc][Bugfix] Disable guided decoding for mistral tokenizer" (#8593) · 02c9afa2
  Roger Wang authored Sep 18, 2024
  
  02c9afa2
- [Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction... · 3118f633
  sroy745 authored Sep 18, 2024
```
[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models.  (#8545)
```
  3118f633
18 Sep, 2024 11 commits
- [Bugfix] add `dead_error` property to engine client (#8574) · 0d47bf3b
  Joe Runde authored Sep 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  0d47bf3b
- [BugFix] Nonzero exit code if MQLLMEngine startup fails (#8572) · d9cd78eb
  Nick Hill authored Sep 18, 2024
  
  d9cd78eb
- [Kernel] Change interface to Mamba selective_state_update for continuous batching (#8039) · db9120cd
  Tyler Michael Smith authored Sep 18, 2024
  
  db9120cd
- [AMD][ROCm]Quantization methods on ROCm; Fix _scaled_mm call (#8380) · b3195bc9
  Gregory Shtrasberg authored Sep 18, 2024
```
Co-authored-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  b3195bc9
- [Model] Support Solar Model (#8386) · e18749ff
  Geun, Lim authored Sep 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e18749ff
- [Core] zmq: bind only to 127.0.0.1 for local-only usage (#8543) · d65798f7
  Russell Bryant authored Sep 18, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  d65798f7
- [Core] *Prompt* logprobs support in Multi-step (#8199) · a8c1d161
  afeldman-nm authored Sep 18, 2024
  
  a8c1d161
- [Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8
  Alexander Matveev authored Sep 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  7c7714d8
- [CI/Build] Update Ruff version (#8469) · 9d104b5b
  Aaron Pham authored Sep 18, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9d104b5b
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
- [Misc] Add argument to disable FastAPI docs (#8554) · e3515729
  Jiaxin Shan authored Sep 18, 2024
  
  e3515729
17 Sep, 2024 3 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [Bugfix] Fix TP > 1 for new granite (#8544) · 98f97133
  Joe Runde authored Sep 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  98f97133
- [Misc] Don't dump contents of kvcache tensors on errors (#8527) · 56c3de01
  Nick Hill authored Sep 17, 2024
  
  56c3de01