Commits · 18e9e1f7b34c46857466fe24e9f9bdee17542f2c · OpenDAS / vllm_cscc · GitLab

13 Sep, 2024 8 commits
- [HotFix] Fix final output truncation with stop string + streaming (#8468) · 18e9e1f7
  Nick Hill authored Sep 13, 2024
  
  18e9e1f7
- [CI/Build] Reorganize models tests (#7820) · a84e598e
  Cyrus Leung authored Sep 14, 2024
  
  a84e598e
- [misc][ci] fix quant test (#8449) · a2469127
  youkaichao authored Sep 13, 2024
  
  a2469127
- [CI/Build] Enable InternVL2 PP test only on single node (#8437) · 9b4a3b23
  Isotr0py authored Sep 13, 2024
  
  9b4a3b23
- [Bugfix] Fix async log stats (#8417) · 68210201
  Alexander Matveev authored Sep 12, 2024
  
  68210201
- [CI/Build] Update pixtral tests to use JSON (#8436) · 84275504
  Cyrus Leung authored Sep 13, 2024
  
  84275504
- [Bugfix] Mapping physical device indices for e2e test utils (#8290) · 40c39653
  shangmingc authored Sep 13, 2024
  
  40c39653
- [Core] Factor out input preprocessing to a separate class (#7329) · 5ec9c0fb
  Cyrus Leung authored Sep 13, 2024
  
  5ec9c0fb
12 Sep, 2024 10 commits
- [Hotfix][Pixtral] Fix multiple images bugs (#8415) · d31174a4
  Patrick von Platen authored Sep 13, 2024
  
  d31174a4
- [CI/Build] Disable multi-node test for InternVL2 (#8428) · b61bd98f
  Roger Wang authored Sep 12, 2024
  
  b61bd98f
- [Core] Add engine option to return only deltas or final output (#7381) · 551ce010
  Nick Hill authored Sep 12, 2024
  
  551ce010
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
- [Bugfix] Offline mode fix (#8376) · f2e263b8
  Joe Runde authored Sep 12, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  f2e263b8
- [Model] Support multiple images for qwen-vl (#8247) · c6202dae
  Alex Brooks authored Sep 12, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c6202dae
- [Bugfix] Fix InternVL2 inference with various num_patches (#8375) · e56bf277
  Isotr0py authored Sep 13, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e56bf277
- [torch.compile] hide slicing under custom op for inductor (#8384) · 7de49aa8
  youkaichao authored Sep 12, 2024
  
  7de49aa8
- [misc] remove engine_use_ray (#8126) · f842a7af
  youkaichao authored Sep 11, 2024
  
  f842a7af
- [MISC] Dump model runner inputs when crashing (#8305) · a65cb160
  Cody Yu authored Sep 11, 2024
  
  a65cb160
11 Sep, 2024 9 commits
- Pixtral (#8377) · d394787e
  Patrick von Platen authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d394787e
- [Speculative Decoding] Test refactor (#8317) · 775f00f8
  Lily Liu authored Sep 11, 2024
```
Co-authored-by: youkaichao <youkaichao@126.com>
```
  775f00f8
- [Kernel][Misc] register ops to prevent graph breaks (#6917) · 73202dbe
  bnellnm authored Sep 11, 2024
```
Co-authored-by: Sage Moore <sage@neuralmagic.com>
```
  73202dbe
- [Hardware][Intel] Support compressed-tensor W8A8 for CPU backend (#7257) · 0b952af4
  Li, Jiang authored Sep 12, 2024
  
  0b952af4
- [Model][VLM] Add Qwen2-VL model support (#7905) · 3b7fea77
  Yang Fan authored Sep 12, 2024
```
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  3b7fea77
- [Frontend] Create ErrorResponse instead of raising exceptions in run_batch (#8347) · cea95dfb
  Pooya Davoodi authored Sep 10, 2024
  
  cea95dfb
- [model] Support for Llava-Next-Video model (#7559) · 6a512a00
  Yangshen⚡Deng authored Sep 11, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
Co-authored-by: Cyrus Leung <tlleungac@connect.ust.hk>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
```
  6a512a00
- [Hardware][NV] Add support for ModelOpt static scaling checkpoints. (#6112) · efcf946a
  Pavani Majety authored Sep 10, 2024
  
  efcf946a
- [Bugfix] Fix InternVL2 vision embeddings process with pipeline parallel (#8299) · 1230263e
  Isotr0py authored Sep 11, 2024
  
  1230263e
10 Sep, 2024 2 commits
- [Frontend] Clean up type annotations for mistral tokenizer (#8314) · 8c054b7a
  Cyrus Leung authored Sep 11, 2024
  
  8c054b7a
- [Misc] Fused MoE Marlin support for GPTQ (#8217) · 6cd5e5b0
  Dipika Sikka authored Sep 09, 2024
  
  6cd5e5b0
09 Sep, 2024 2 commits
- [Misc] GPTQ Activation Ordering (#8135) · c7cb5c33
  Kyle Sayers authored Sep 09, 2024
  
  c7cb5c33
- [Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures... · 08287ef6
  Kyle Mistele authored Sep 09, 2024
```
[Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures Vercel AI SDK compatibility (#8272)
```
  08287ef6
07 Sep, 2024 5 commits
- [CI/Build] Use python 3.12 in cuda image (#8133) · cfe712bf
  Joe Runde authored Sep 07, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  cfe712bf
- [Model][VLM] Support multi-images inputs for InternVL2 models (#8201) · e8071259
  Isotr0py authored Sep 07, 2024
  
  e8071259
- [Bugfix] Fix broken OpenAI tensorizer test (#8258) · 9f68e00d
  Cyrus Leung authored Sep 07, 2024
  
  9f68e00d
- [tpu][misc] fix typo (#8260) · ce2702a9
  youkaichao authored Sep 06, 2024
  
  ce2702a9
- [Model] Multi-input support for LLaVA (#8238) · 2f707fcb
  Cyrus Leung authored Sep 07, 2024
  
  2f707fcb
06 Sep, 2024 4 commits
- [Model] Allow loading from original Mistral format (#8168) · 29f49cd6
  Patrick von Platen authored Sep 07, 2024
```
Co-authored-by: Michael Goin <michael@neuralmagic.com>
```
  29f49cd6
- [CI/Build] Increasing timeout for multiproc worker tests (#8203) · 1447c97e
  Alexey Kondratiev(AMD) authored Sep 06, 2024
  
  1447c97e
- [Frontend] Add --logprobs argument to `benchmark_serving.py` (#8191) · e5cab715
  afeldman-nm authored Sep 06, 2024
  
  e5cab715
- [Core] Support load and unload LoRA in api server (#6566) · db3bf7c9
  Jiaxin Shan authored Sep 05, 2024
```
Co-authored-by: Jee Jee Li <pandaleefree@gmail.com>
```
  db3bf7c9