Commits · ee37328da085af14f89ad1af8eb2c359ae2f46a1 · OpenDAS / vllm_cscc

"tests/vscode:/vscode.git/clone" did not exist on "8a7fe47d322920bdff1b1c3472fe7f423a73a23b"

30 Apr, 2024 8 commits
- fix_tokenizer_snapshot_download_bug (#4493) · 6ad58f42
  fuchen.ljl authored May 01, 2024
  
  6ad58f42
- [Bugfix][Minor] Make ignore_eos effective (#4468) · dd1a50a8
  Li, Jiang authored May 01, 2024
  
  dd1a50a8
- [Frontend] [Core] Tensorizer: support dynamic `num_readers`, update version (#4467) · 715c2d85
  Alpay Ariyak authored Apr 30, 2024
  
  715c2d85
- [Frontend] Support complex message content for chat completions endpoint (#3467) · a4941404
  Florian Greinacher authored May 01, 2024
```
Co-authored-by: Lily Liu <lilyliupku@gmail.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
```
  a4941404
- [Kernel] Support Fp8 Checkpoints (Dynamic + Static) (#4332) · 111815d4
  Robert Shaw authored Apr 30, 2024
```
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com>
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
Co-authored-by: mgoin <michael@neuralmagic.com>
Co-authored-by: Tyler Michael Smith <tyler@neuralmagic.com>
Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>
```
  111815d4
- [BugFix] fix num_lookahead_slots missing in async executor (#4165) · 4bb53e2d
  leiwen83 authored May 01, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  4bb53e2d
- [Core]Refactor gptq_marlin ops (#4466) · 26f2fb51
  Kunshang Ji authored Apr 30, 2024
  
  26f2fb51
- [Bugfix][Kernel] Fix compute_type for MoE kernel (#4463) · fa322078
  Woosuk Kwon authored Apr 29, 2024
  
  fa322078
29 Apr, 2024 3 commits
- [Core][Distributed] use cpu group to broadcast metadata in cpu (#4444) · f4f921b7
  youkaichao authored Apr 29, 2024
  
  f4f921b7
- [Kernel] Marlin Expansion: Support AutoGPTQ Models with Marlin (#3922) · 73c8d677
  Robert Shaw authored Apr 29, 2024
```
Co-authored-by: alexm <alexm@neuralmagic.com>
Co-authored-by: mgoin <michael@neuralmagic.com>
```
  73c8d677
- [mypy][5/N] Support all typing on model executor (#4427) · df29793d
  SangBin Cho authored Apr 29, 2024
  
  df29793d
28 Apr, 2024 2 commits
- Add more Prometheus metrics (#2764) · bf480c53
  Ronen Schaffer authored Apr 29, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  bf480c53
- [Misc] fix typo in llm_engine init logging (#4428) · 9c7306ac
  DefTruth authored Apr 28, 2024
  
  9c7306ac
27 Apr, 2024 13 commits
- [BugFix] Resolved Issues For LinearMethod --> QuantConfig (#4418) · 4ea1f967
  Robert Shaw authored Apr 27, 2024
  
  4ea1f967
- [BugFix] Fix return type of executor execute_model methods (#4402) · ba4be44c
  Nick Hill authored Apr 27, 2024
  
  ba4be44c
- [Core] Support offline use of local cache for models (#4374) · d6e520e1
  Prashant Gupta authored Apr 27, 2024
```
Signed-off-by: Prashant Gupta <prashantgupta@us.ibm.com>
Co-authored-by: Travis Johnson <tjohnson31415@gmail.com>
```
  d6e520e1
- [BugFix] Fix `min_tokens` when `eos_token_id` is None (#4389) · 81661da7
  Nick Hill authored Apr 27, 2024
```
Co-authored-by: DefTruth <31974251+deftruth@users.noreply.github.com>
```
  81661da7
- [Bugfix] Abort requests when the connection to /v1/completions is interrupted (#4363) · dfea1731
  Ruoyu Qin authored Apr 28, 2024
  
  dfea1731
- [Bugfix][Core] Fix get decoding config from ray (#4335) · 7134303c
  Roy authored Apr 27, 2024
  
  7134303c
- [Model] Phi-3 4k sliding window temp. fix (#4380) · 3da24c2d
  Caio Mendes authored Apr 27, 2024
  
  3da24c2d
- [Kernel] Full Tensor Parallelism for LoRA Layers (#3524) · eefeb164
  Austin Veselka authored Apr 27, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  eefeb164
- [ROCm][Hardware][AMD] Enable group query attention for triton FA (#4406) · 18d23f64
  Hongxia Yang authored Apr 27, 2024
  
  18d23f64
- [Misc] Fix logger format typo (#4396) · 87f545ba
  Roy authored Apr 27, 2024
  
  87f545ba
- [Frontend][Bugfix] Disallow extra fields in OpenAI API (#4355) · 8947bc3c
  Cyrus Leung authored Apr 27, 2024
  
  8947bc3c
- [Kernel] Optimize FP8 support for MoE kernel / Mixtral via static scales (#4343) · 12628d3c
  Philipp Moritz authored Apr 26, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  12628d3c
- [Core] Introduce `DistributedGPUExecutor` abstract class (#4348) · 258a2c58
  Nick Hill authored Apr 26, 2024
  
  258a2c58
26 Apr, 2024 5 commits
- [Misc][Refactor] Generalize linear_method to be quant_method (#4373) · a62aaf1d
  Cody Yu authored Apr 26, 2024
  
  a62aaf1d
- [Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309) · 603ad848
  SangBin Cho authored Apr 26, 2024
  
  603ad848
- [CI] Disable non-lazy string operation on logging (#4326) · a88081bf
  SangBin Cho authored Apr 26, 2024
```
Co-authored-by: Danny Guinther <dguinther@neuralmagic.com>
```
  a88081bf
- [Frontend] Add --log-level option to api server (#4377) · 2f30e7c7
  Norman Mu authored Apr 25, 2024
  
  2f30e7c7
- [Bugfix] Fix parameter name in `get_tokenizer` (#4107) · a74dee9b
  Cyrus Leung authored Apr 26, 2024
  
  a74dee9b
25 Apr, 2024 8 commits
- [Core] Move function tracing setup to util function (#4352) · efffb63f
  Nick Hill authored Apr 25, 2024
  
  efffb63f
- [Core] Add `shutdown()` method to `ExecutorBase` (#4349) · 15e7c675
  Nick Hill authored Apr 25, 2024
  
  15e7c675
- [Misc] Fix flash attention backend log (#4368) · b6dcb4d4
  Roy authored Apr 26, 2024
  
  b6dcb4d4
- [Mypy] Typing lora folder (#4337) · b5b4a398
  SangBin Cho authored Apr 26, 2024
  
  b5b4a398
- [Core]refactor aqlm quant ops (#4351) · f4bc4de1
  Kunshang Ji authored Apr 25, 2024
  
  f4bc4de1
- [Bugfix][Model] Refactor OLMo model to support new HF format in transformers 4.40.0 (#4324) · fbf152d9
  Isotr0py authored Apr 26, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  fbf152d9
- [Core] Move ray_utils.py from `engine` to `executor` package (#4347) · 479d69fa
  Nick Hill authored Apr 24, 2024
  
  479d69fa
- [Model] Adds Phi-3 support (#4298) · 96e90fde
  Caio Mendes authored Apr 25, 2024
  
  96e90fde
24 Apr, 2024 1 commit
- [Core][Distributed] use existing torch.cuda.device (#4318) · 3cd9b5bb
  youkaichao authored Apr 24, 2024
```
[Core][Distributed] use existing torch.cuda.device context manager (#4318)
```
  3cd9b5bb