Commits · 18ae428a0d8792d160d811a9cd5bb004d68ea8bd · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "05972ea7e5f81250cc4ceaae8a174cfffe7755ac"

20 Sep, 2024 1 commit
- [Bugfix] Fix Phi3.5 mini and MoE LoRA inference (#8571) · 18ae428a
  Amit Garg authored Sep 19, 2024
  
  18ae428a
19 Sep, 2024 5 commits
- [Core] simplify logits resort in _apply_top_k_top_p (#8619) · e42c634a
  盏一 authored Sep 20, 2024
  
  e42c634a
- [Kernel][Amd] Add fp8 kv cache support for rocm custom paged attention (#8577) · 9cc373f3
  Charlie Fu authored Sep 19, 2024
  
  9cc373f3
- [Frontend] Use MQLLMEngine for embeddings models too (#8584) · 76515f30
  Nick Hill authored Sep 19, 2024
  
  76515f30
- Revert "[Misc][Bugfix] Disable guided decoding for mistral tokenizer" (#8593) · 02c9afa2
  Roger Wang authored Sep 18, 2024
  
  02c9afa2
- [Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction... · 3118f633
  sroy745 authored Sep 18, 2024
```
[Bugfix] [Encoder-Decoder] Bugfix for encoder specific metadata construction during decode of encoder-decoder models.  (#8545)
```
  3118f633
18 Sep, 2024 11 commits
- [Bugfix] add `dead_error` property to engine client (#8574) · 0d47bf3b
  Joe Runde authored Sep 18, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  0d47bf3b
- [BugFix] Nonzero exit code if MQLLMEngine startup fails (#8572) · d9cd78eb
  Nick Hill authored Sep 18, 2024
  
  d9cd78eb
- [Kernel] Change interface to Mamba selective_state_update for continuous batching (#8039) · db9120cd
  Tyler Michael Smith authored Sep 18, 2024
  
  db9120cd
- [AMD][ROCm]Quantization methods on ROCm; Fix _scaled_mm call (#8380) · b3195bc9
  Gregory Shtrasberg authored Sep 18, 2024
```
Co-authored-by: Alexei-V-Ivanov-AMD <156011006+Alexei-V-Ivanov-AMD@users.noreply.github.com>
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  b3195bc9
- [Model] Support Solar Model (#8386) · e18749ff
  Geun, Lim authored Sep 19, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  e18749ff
- [Core] zmq: bind only to 127.0.0.1 for local-only usage (#8543) · d65798f7
  Russell Bryant authored Sep 18, 2024
```
Signed-off-by: Russell Bryant <rbryant@redhat.com>
```
  d65798f7
- [Core] *Prompt* logprobs support in Multi-step (#8199) · a8c1d161
  afeldman-nm authored Sep 18, 2024
  
  a8c1d161
- [Core][Bugfix][Perf] Introduce `MQLLMEngine` to avoid `asyncio` OH (#8157) · 7c7714d8
  Alexander Matveev authored Sep 18, 2024
```
Co-authored-by: Nick Hill <nickhill@us.ibm.com>
Co-authored-by: rshaw@neuralmagic.com <rshaw@neuralmagic.com>
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  7c7714d8
- [CI/Build] Update Ruff version (#8469) · 9d104b5b
  Aaron Pham authored Sep 18, 2024
```
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  9d104b5b
- [CI/Build] Avoid CUDA initialization (#8534) · 6ffa3f31
  Cyrus Leung authored Sep 18, 2024
  
  6ffa3f31
- [Misc] Add argument to disable FastAPI docs (#8554) · e3515729
  Jiaxin Shan authored Sep 18, 2024
  
  e3515729
17 Sep, 2024 11 commits
- [Kernel] Change interface to Mamba causal_conv1d_update for continuous batching (#8012) · 8110e445
  Tyler Michael Smith authored Sep 17, 2024
  
  8110e445
- [Bugfix] Fix TP > 1 for new granite (#8544) · 98f97133
  Joe Runde authored Sep 17, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  98f97133
- [Misc] Don't dump contents of kvcache tensors on errors (#8527) · 56c3de01
  Nick Hill authored Sep 17, 2024
  
  56c3de01
- [Model] Add mistral function calling format to all models loaded with "mistral" format (#8515) · a54ed802
  Patrick von Platen authored Sep 17, 2024
```
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  a54ed802
- [Feature][kernel] tensor parallelism with bitsandbytes quantization (#8434) · 9855b995
  chenqianfzh authored Sep 17, 2024
  
  9855b995
- [Encoder decoder] Add cuda graph support during decoding for encoder-decoder models (#7631) · 1009e93c
  sroy745 authored Sep 17, 2024
  
  1009e93c
- [Misc] Limit to ray[adag] 2.35 to avoid backward incompatible change (#8509) · cbdb2522
  Rui Qiao authored Sep 17, 2024
```
Signed-off-by: Rui Qiao <ruisearch42@gmail.com>
```
  cbdb2522
- [torch.compile] register allreduce operations as custom ops (#8526) · 99aa4edd
  youkaichao authored Sep 16, 2024
  
  99aa4edd
- [Misc][Bugfix] Disable guided decoding for mistral tokenizer (#8521) · ee2bceaa
  Roger Wang authored Sep 16, 2024
  
  ee2bceaa
- [Frontend] Improve Nullable kv Arg Parsing (#8525) · 1c1bb388
  Alex Brooks authored Sep 16, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
```
  1c1bb388
- [refactor] remove triton based sampler (#8524) · 546034b4
  Simon Mo authored Sep 16, 2024
  
  546034b4
16 Sep, 2024 5 commits
- [Bugfix] Bind api server port before starting engine (#8491) · 47f5e03b
  Kevin Lin authored Sep 16, 2024
  
  47f5e03b
- [Kernel] AQ AZP 3/4: Asymmetric quantization kernels (#7270) · 5d73ae49
  Luka Govedič authored Sep 16, 2024
  
  5d73ae49
- [BugFix] Fix clean shutdown issues (#8492) · acd5511b
  Nick Hill authored Sep 16, 2024
  
  acd5511b
- [Frontend] Expose revision arg in OpenAI server (#8501) · 837c1968
  lewtun authored Sep 16, 2024
  
  837c1968
- [Kernel] Enable 8-bit weights in Fused Marlin MoE (#8032) · a091e2da
  ElizaWszola authored Sep 16, 2024
```
Co-authored-by: Dipika <dipikasikka1@gmail.com>
```
  a091e2da
15 Sep, 2024 2 commits
- [Bugfix][Kernel] Add `IQ1_M` quantization implementation to GGUF kernel (#8357) · fc990f97
  Isotr0py authored Sep 16, 2024
  
  fc990f97
- [Bugfix][Model] Fix Python 3.8 compatibility in Pixtral model by updating type annotations (#8490) · 3724d5f6
  Chris authored Sep 15, 2024
  
  3724d5f6
14 Sep, 2024 5 commits
- [TPU] Implement multi-step scheduling (#8489) · 50e9ec41
  Woosuk Kwon authored Sep 14, 2024
  
  50e9ec41
- [torch.compile] add a flag to disable custom op (#8488) · 47790f3e
  youkaichao authored Sep 14, 2024
  
  47790f3e
- [torch.compile] fix functionalization (#8480) · a36e070d
  youkaichao authored Sep 14, 2024
  
  a36e070d
- [Model] support minicpm3 (#8297) · 8a0cf1dd
  ywfang authored Sep 14, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  8a0cf1dd
- [Kernel][Hardware][Amd]Custom paged attention kernel for rocm (#8310) · 1ef0d2ef
  Charlie Fu authored Sep 13, 2024
  
  1ef0d2ef