Commits · a88bb9b032d75aad74b2e1bd3d97b8e8a24e8b9d · OpenDAS / vllm_cscc · GitLab

01 May, 2024 8 commits
- [Bugfix] Fix the fp8 kv_cache check error that occurs when failing to obtain... · a88bb9b0
  AnyISalIn authored May 02, 2024
```
[Bugfix] Fix the fp8 kv_cache check error that occurs when failing to obtain the CUDA version. (#4173)
Signed-off-by: AnyISalIn <anyisalin@gmail.com>
```
  a88bb9b0
- [Test] Add ignore_eos test (#4519) · 6f1df804
  SangBin Cho authored May 01, 2024
  
  6f1df804
- [Misc]Add customized information for models (#4132) · d6f4bd7c
  Jee Li authored May 01, 2024
  
  d6f4bd7c
- Allow user to define whitespace pattern for outlines (#4305) · c3845d82
  Robert Caulk authored May 01, 2024
  
  c3845d82
- [Misc] fix typo in block manager (#4453) · a822eb34
  Pastel！ authored May 01, 2024
  
  a822eb34
- [Misc][Typo] type annotation fix (#4495) · f458112e
  harrywu authored May 01, 2024
  
  f458112e
- [Core] Centralize GPU Worker construction (#4419) · 2e240c69
  Nick Hill authored Apr 30, 2024
  
  2e240c69
- Unable to find Punica extension issue during source code installation (#4494) · ee37328d
  fuchen.ljl authored May 01, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  ee37328d
30 Apr, 2024 10 commits
- fix_tokenizer_snapshot_download_bug (#4493) · 6ad58f42
  fuchen.ljl authored May 01, 2024
  
  6ad58f42
- [Bugfix][Minor] Make ignore_eos effective (#4468) · dd1a50a8
  Li, Jiang authored May 01, 2024
  
  dd1a50a8
- [Frontend] [Core] Tensorizer: support dynamic `num_readers`, update version (#4467) · 715c2d85
  Alpay Ariyak authored Apr 30, 2024
  
  715c2d85
- [Frontend] Support complex message content for chat completions endpoint (#3467) · a4941404
  Florian Greinacher authored May 01, 2024
```
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  a4941404
- [Kernel] Support Fp8 Checkpoints (Dynamic + Static) (#4332) · 111815d4
  Robert Shaw authored Apr 30, 2024
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  111815d4
- [Doc] add visualization for multi-stage dockerfile (#4456) · b31a1fb6
  Prashant Gupta authored Apr 30, 2024
```
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  b31a1fb6
- [BugFix] fix num_lookahead_slots missing in async executor (#4165) · 4bb53e2d
  leiwen83 authored May 01, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  4bb53e2d
- [Core]Refactor gptq_marlin ops (#4466) · 26f2fb51
  Kunshang Ji authored Apr 30, 2024
  
  26f2fb51
- [Bugfix][Kernel] Fix compute_type for MoE kernel (#4463) · fa322078
  Woosuk Kwon authored Apr 29, 2024
  
  fa322078
- [Misc] Upgrade to `torch==2.3.0` (#4454) · d627a3d8
  Michael Goin authored Apr 29, 2024
  
  d627a3d8
29 Apr, 2024 4 commits
- [Core][Distributed] use cpu group to broadcast metadata in cpu (#4444) · f4f921b7
  youkaichao authored Apr 29, 2024
  
  f4f921b7
- [CI] hotfix: soft fail neuron test (#4458) · ac5ccf01
  Simon Mo authored Apr 29, 2024
  
  ac5ccf01
- [Kernel] Marlin Expansion: Support AutoGPTQ Models with Marlin (#3922) · 73c8d677
  Robert Shaw authored Apr 29, 2024
```
Co-authored-by: alexm <alexm@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  73c8d677
- [mypy][5/N] Support all typing on model executor (#4427) · df29793d
  SangBin Cho authored Apr 29, 2024
  
  df29793d
28 Apr, 2024 3 commits
- [CI] clean docker cache for neuron (#4441) · 03dd7d52
  Simon Mo authored Apr 28, 2024
  
  03dd7d52
- Add more Prometheus metrics (#2764) · bf480c53
  Ronen Schaffer authored Apr 29, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  bf480c53
- [Misc] fix typo in llm_engine init logging (#4428) · 9c7306ac
  DefTruth authored Apr 28, 2024
  
  9c7306ac
27 Apr, 2024 13 commits
- [BugFix] Resolved Issues For LinearMethod --> QuantConfig (#4418) · 4ea1f967
  Robert Shaw authored Apr 27, 2024
  
  4ea1f967
- [BugFix] Fix return type of executor execute_model methods (#4402) · ba4be44c
  Nick Hill authored Apr 27, 2024
  
  ba4be44c
- [Core] Support offline use of local cache for models (#4374) · d6e520e1
  Prashant Gupta authored Apr 27, 2024
```
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Travis Johnson <tjohnson31415@gmail.com>
```
  d6e520e1
- [BugFix] Fix `min_tokens` when `eos_token_id` is None (#4389) · 81661da7
  Nick Hill authored Apr 27, 2024
```
Co-authored-by: DefTruth <31974251+deftruth@users.noreply.github.com>
```
  81661da7
- [Bugfix] Abort requests when the connection to /v1/completions is interrupted (#4363) · dfea1731
  Ruoyu Qin authored Apr 28, 2024
  
  dfea1731
- [Bugfix][Core] Fix get decoding config from ray (#4335) · 7134303c
  Roy authored Apr 27, 2024
  
  7134303c
- [Model] Phi-3 4k sliding window temp. fix (#4380) · 3da24c2d
  Caio Mendes authored Apr 27, 2024
  
  3da24c2d
- [Kernel] Full Tensor Parallelism for LoRA Layers (#3524) · eefeb164
  Austin Veselka authored Apr 27, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  eefeb164
- [ROCm][Hardware][AMD] Enable group query attention for triton FA (#4406) · 18d23f64
  Hongxia Yang authored Apr 27, 2024
  
  18d23f64
- [Misc] Fix logger format typo (#4396) · 87f545ba
  Roy authored Apr 27, 2024
  
  87f545ba
- [Frontend][Bugfix] Disallow extra fields in OpenAI API (#4355) · 8947bc3c
  Cyrus Leung authored Apr 27, 2024
  
  8947bc3c
- [Kernel] Optimize FP8 support for MoE kernel / Mixtral via static scales (#4343) · 12628d3c
  Philipp Moritz authored Apr 26, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  12628d3c
- [Core] Introduce `DistributedGPUExecutor` abstract class (#4348) · 258a2c58
  Nick Hill authored Apr 26, 2024
  
  258a2c58
26 Apr, 2024 2 commits
- [Misc] add RFC issue template (#4401) · aba47be3
  youkaichao authored Apr 26, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  aba47be3
- [Misc][Refactor] Generalize linear_method to be quant_method (#4373) · a62aaf1d
  Cody Yu authored Apr 26, 2024
  
  a62aaf1d