Commits · a494140433be496a0321999955acf7e6387986b3 · OpenDAS / vllm_cscc · GitLab

30 Apr, 2024 7 commits
- [Frontend] Support complex message content for chat completions endpoint (#3467) · a4941404
  Florian Greinacher authored May 01, 2024
```
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  a4941404
- [Kernel] Support Fp8 Checkpoints (Dynamic + Static) (#4332) · 111815d4
  Robert Shaw authored Apr 30, 2024
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  111815d4
- [Doc] add visualization for multi-stage dockerfile (#4456) · b31a1fb6
  Prashant Gupta authored Apr 30, 2024
```
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b31a1fb6
- [BugFix] fix num_lookahead_slots missing in async executor (#4165) · 4bb53e2d
  leiwen83 authored May 01, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  4bb53e2d
- [Core]Refactor gptq_marlin ops (#4466) · 26f2fb51
  Kunshang Ji authored Apr 30, 2024
  
  26f2fb51
- [Bugfix][Kernel] Fix compute_type for MoE kernel (#4463) · fa322078
  Woosuk Kwon authored Apr 29, 2024
  
  fa322078
- [Misc] Upgrade to `torch==2.3.0` (#4454) · d627a3d8
  Michael Goin authored Apr 29, 2024
  
  d627a3d8
29 Apr, 2024 4 commits
- [Core][Distributed] use cpu group to broadcast metadata in cpu (#4444) · f4f921b7
  youkaichao authored Apr 29, 2024
  
  f4f921b7
- [CI] hotfix: soft fail neuron test (#4458) · ac5ccf01
  Simon Mo authored Apr 29, 2024
  
  ac5ccf01
- [Kernel] Marlin Expansion: Support AutoGPTQ Models with Marlin (#3922) · 73c8d677
  Robert Shaw authored Apr 29, 2024
```
Co-authored-by: alexm <alexm@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  73c8d677
- [mypy][5/N] Support all typing on model executor (#4427) · df29793d
  SangBin Cho authored Apr 29, 2024
  
  df29793d
28 Apr, 2024 3 commits
- [CI] clean docker cache for neuron (#4441) · 03dd7d52
  Simon Mo authored Apr 28, 2024
  
  03dd7d52
- Add more Prometheus metrics (#2764) · bf480c53
  Ronen Schaffer authored Apr 29, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  bf480c53
- [Misc] fix typo in llm_engine init logging (#4428) · 9c7306ac
  DefTruth authored Apr 28, 2024
  
  9c7306ac
27 Apr, 2024 13 commits
- [BugFix] Resolved Issues For LinearMethod --> QuantConfig (#4418) · 4ea1f967
  Robert Shaw authored Apr 27, 2024
  
  4ea1f967
- [BugFix] Fix return type of executor execute_model methods (#4402) · ba4be44c
  Nick Hill authored Apr 27, 2024
  
  ba4be44c
- [Core] Support offline use of local cache for models (#4374) · d6e520e1
  Prashant Gupta authored Apr 27, 2024
```
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Travis Johnson <tjohnson31415@gmail.com>
```
  d6e520e1
- [BugFix] Fix `min_tokens` when `eos_token_id` is None (#4389) · 81661da7
  Nick Hill authored Apr 27, 2024
```
Co-authored-by: DefTruth <31974251+deftruth@users.noreply.github.com>
```
  81661da7
- [Bugfix] Abort requests when the connection to /v1/completions is interrupted (#4363) · dfea1731
  Ruoyu Qin authored Apr 28, 2024
  
  dfea1731
- [Bugfix][Core] Fix get decoding config from ray (#4335) · 7134303c
  Roy authored Apr 27, 2024
  
  7134303c
- [Model] Phi-3 4k sliding window temp. fix (#4380) · 3da24c2d
  Caio Mendes authored Apr 27, 2024
  
  3da24c2d
- [Kernel] Full Tensor Parallelism for LoRA Layers (#3524) · eefeb164
  Austin Veselka authored Apr 27, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  eefeb164
- [ROCm][Hardware][AMD] Enable group query attention for triton FA (#4406) · 18d23f64
  Hongxia Yang authored Apr 27, 2024
  
  18d23f64
- [Misc] Fix logger format typo (#4396) · 87f545ba
  Roy authored Apr 27, 2024
  
  87f545ba
- [Frontend][Bugfix] Disallow extra fields in OpenAI API (#4355) · 8947bc3c
  Cyrus Leung authored Apr 27, 2024
  
  8947bc3c
- [Kernel] Optimize FP8 support for MoE kernel / Mixtral via static scales (#4343) · 12628d3c
  Philipp Moritz authored Apr 26, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  12628d3c
- [Core] Introduce `DistributedGPUExecutor` abstract class (#4348) · 258a2c58
  Nick Hill authored Apr 26, 2024
  
  258a2c58
26 Apr, 2024 7 commits
- [Misc] add RFC issue template (#4401) · aba47be3
  youkaichao authored Apr 26, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  aba47be3
- [Misc][Refactor] Generalize linear_method to be quant_method (#4373) · a62aaf1d
  Cody Yu authored Apr 26, 2024
  
  a62aaf1d
- [Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309) · 603ad848
  SangBin Cho authored Apr 26, 2024
  
  603ad848
- [CI] Disable non-lazy string operation on logging (#4326) · a88081bf
  SangBin Cho authored Apr 26, 2024
```
Co-authored-by: Danny Guinther <dguinther@neuralmagic.com>
```
  a88081bf
- [Frontend] Add --log-level option to api server (#4377) · 2f30e7c7
  Norman Mu authored Apr 25, 2024
  
  2f30e7c7
- [Bugfix] Fix parameter name in `get_tokenizer` (#4107) · a74dee9b
  Cyrus Leung authored Apr 26, 2024
  
  a74dee9b
- [ROCm][Hardware][AMD][Doc] Documentation update for ROCm (#4376) · cf29b7ed
  Hongxia Yang authored Apr 25, 2024
```
Co-authored-by: WoosukKwon <woosuk.kwon@berkeley.edu>
```
  cf29b7ed
25 Apr, 2024 6 commits
- [Core] Move function tracing setup to util function (#4352) · efffb63f
  Nick Hill authored Apr 25, 2024
  
  efffb63f
- [Core] Add `shutdown()` method to `ExecutorBase` (#4349) · 15e7c675
  Nick Hill authored Apr 25, 2024
  
  15e7c675
- [Misc] Fix flash attention backend log (#4368) · b6dcb4d4
  Roy authored Apr 26, 2024
  
  b6dcb4d4
- [Mypy] Typing lora folder (#4337) · b5b4a398
  SangBin Cho authored Apr 26, 2024
  
  b5b4a398
- [Core]refactor aqlm quant ops (#4351) · f4bc4de1
  Kunshang Ji authored Apr 25, 2024
  
  f4bc4de1
- [Doc] README Phi-3 name fix. (#4372) · bd7a8eef
  Caio Mendes authored Apr 25, 2024
```
Co-authored-by: Caio Mendes <caiocesart@microsoft.com>
```
  bd7a8eef