Commits · 13f9f7a3d0373421ee9fd7498e450214e134aa6c · OpenDAS / vllm_cscc

"vllm/engine/async_llm_engine.py" did not exist on "da5ddcd544ac5ce6bc4f522af9cbdc315f94620e"

25 Sep, 2024 1 commit
- [[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768) · 13f9f7a3
  Jee Jee Li authored Sep 25, 2024
  
  13f9f7a3
24 Sep, 2024 5 commits
- [Frontend] Batch inference for llm.chat() API (#8648) · 2529d09b
  Andy authored Sep 24, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  2529d09b
- [Model] Expose Phi3v num_crops as a mm_processor_kwarg (#8658) · 8ff7ced9
  Alex Brooks authored Sep 24, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8ff7ced9
- Revert "[Core] Rename `PromptInputs` to `PromptType`, and `inputs` to `prompt`" (#8750) · 3185fb0c
  Simon Mo authored Sep 23, 2024
  
  3185fb0c
- re-implement beam search on top of vllm core (#8726) · 0250dd68
  youkaichao authored Sep 23, 2024
```
Co-authored-by: Brendan Wong <bjwpokemon@gmail.com>
```
  0250dd68
- Fix tests in test_scheduler.py that fail with BlockManager V2 (#8728) · 88577ac9
  sroy745 authored Sep 23, 2024
  
  88577ac9
23 Sep, 2024 7 commits
- Add output streaming support to multi-step + async while ensuring RequestOutput obj reuse (#8335) · 1a2aef3e
  Alexander Matveev authored Sep 23, 2024
  
  1a2aef3e
- Fix typical acceptance sampler with correct recovered token ids (#8562) · 5f7bb584
  jiqing-feng authored Sep 24, 2024
  
  5f7bb584
- [Kernel][LoRA] Add assertion for punica sgmv kernels (#7585) · 9b0e3ec9
  Jee Jee Li authored Sep 24, 2024
  
  9b0e3ec9
- [Kernel] (2/N) Machete - Integrate into CompressedTensorsWNA16 and GPTQMarlin (#7701) · 86e9c8df
  Lucas Wilkinson authored Sep 23, 2024
```
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Divakar Verma <137818590+divakar-amd@users.noreply.github.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
```
  86e9c8df
- [CI/Build] use setuptools-scm to set __version__ (#4738) · ee5f34b1
  Daniele authored Sep 23, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  ee5f34b1
- [Model] Support pp for qwen2-vl (#8696) · a79e5229
  Yanyi Liu authored Sep 23, 2024
  
  a79e5229
- [Core][Frontend] Support Passing Multimodal Processor Kwargs (#8657) · 9b8c8ba1
  Alex Brooks authored Sep 23, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  9b8c8ba1
22 Sep, 2024 2 commits

[SpecDec][Misc] Cleanup, remove bonus token logic. (#8701) · c6bd70d7
Lily Liu authored Sep 22, 2024

c6bd70d7

[Model][VLM] Add LLaVA-Onevision model support (#8486) · 5b595327

litianjian authored Sep 23, 2024


Co-authored-by: litianjian <litianjian@bytedance.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>

5b595327

21 Sep, 2024 3 commits
- [beam search] add output for manually checking the correctness (#8684) · 0faab90e
  youkaichao authored Sep 20, 2024
  
  0faab90e
- [Core] Factor out common code in `SequenceData` and `Sequence` (#8675) · 0455c46e
  Cyrus Leung authored Sep 21, 2024
  
  0455c46e
- [Core] Rename `PromptInputs` and `inputs`(#8673) · 0057894e
  Cyrus Leung authored Sep 21, 2024
  
  0057894e
20 Sep, 2024 2 commits
- [Bugfix][Core] Fix tekken edge case for mistral tokenizer (#8640) · b4e4eda9
  Patrick von Platen authored Sep 20, 2024
  
  b4e4eda9
- [Core] Support Lora lineage and base model metadata management (#6315) · 260d40b5
  Jiaxin Shan authored Sep 19, 2024
  
  260d40b5
19 Sep, 2024 2 commits
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
- [Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction... · 3118f633
  sroy745 authored Sep 18, 2024
```
[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models.  (#8545)
```
  3118f633
18 Sep, 2024 5 commits
- [Kernel] Change interface to Mamba selective_state_update for continuous batching (#8039) · db9120cd
  Tyler Michael Smith authored Sep 18, 2024
  
  db9120cd
- [Core] *Prompt* logprobs support in Multi-step (#8199) · a8c1d161
  afeldman-nm authored Sep 18, 2024
  
  a8c1d161
- [Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8
  Alexander Matveev authored Sep 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  7c7714d8
- [CI/Build] Update Ruff version (#8469) · 9d104b5b
  Aaron Pham authored Sep 18, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9d104b5b
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
17 Sep, 2024 8 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [doc] improve installation doc (#8550) · fa0c114f
  youkaichao authored Sep 17, 2024
```
Co-authored-by: Andy Dai <76841985+Imss27@users.noreply.github.com>
```
  fa0c114f
- [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) · a54ed802
  Patrick von Platen authored Sep 17, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a54ed802
- [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) · 9855b995
  chenqianfzh authored Sep 17, 2024
  
  9855b995
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
- [torch.compile] register allreduce operations as custom ops (#8526) · 99aa4edd
  youkaichao authored Sep 16, 2024
  
  99aa4edd
- [Frontend] Improve Nullable kv Arg Parsing (#8525) · 1c1bb388
  Alex Brooks authored Sep 16, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  1c1bb388
- [refactor] remove triton based sampler (#8524) · 546034b4
  Simon Mo authored Sep 16, 2024
  
  546034b4
16 Sep, 2024 3 commits
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [BugFix] Fix clean shutdown issues (#8492) · acd5511b
  Nick Hill authored Sep 16, 2024
  
  acd5511b
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 1 commit
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
14 Sep, 2024 1 commit
- [torch.compile] add a flag to disable custom op (#8488) · 47790f3e
  youkaichao authored Sep 14, 2024
  
  47790f3e