Commits · 160e1d8c99567872ef3e683e87c09f96a3a2e7c4 · OpenDAS / vllm_cscc

"vscode:/vscode.git/clone" did not exist on "d3c84297c38c79e2da6cd0a4ba8be58d95d13d97"

16 Jul, 2024 3 commits
- [Hardware][TPU] Support MoE with Pallas GMM kernel (#6457) · c467dff2
  Woosuk Kwon authored Jul 16, 2024
  
  c467dff2
- [Core] Use numpy to speed up padded token processing (#6442) · 2bb0489c
  Peng Guanwen authored Jul 16, 2024
  
  2bb0489c
- [BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425) · 9ad32dac
  Mor Zusman authored Jul 16, 2024
```
Co-authored-by: Mor Zusman <morz@ai21.com>
```
  9ad32dac
15 Jul, 2024 5 commits
- [Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289) · ec9933f4
  Woosuk Kwon authored Jul 15, 2024
  
  ec9933f4
- [misc][distributed] fix pp missing layer condition (#6446) · 4cf256ae
  youkaichao authored Jul 15, 2024
  
  4cf256ae
- [Kernel] Use CUTLASS kernels for the FP8 layers with Bias (#6270) · c8fd97f2
  Tyler Michael Smith authored Jul 15, 2024
  
  c8fd97f2
- [VLM] Minor space optimization for `ClipVisionModel` (#6436) · 6ae1597d
  Roger Wang authored Jul 15, 2024
  
  6ae1597d
- [core][distributed] simplify code to support pipeline parallel (#6406) · 69672f11
  youkaichao authored Jul 14, 2024
  
  69672f11
14 Jul, 2024 3 commits
- [ Misc ] Enable Quantizing All Layers of DeekSeekv2 (#6423) · 73030b7d
  Robert Shaw authored Jul 14, 2024
  
  73030b7d
- [Model] Initialize Fuyu-8B support (#3924) · 540c0368
  Isotr0py authored Jul 14, 2024
```
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  540c0368
- [ Misc ] Apply MoE Refactor to Deepseekv2 To Support Fp8 (#6417) · fb6af8bc
  Robert Shaw authored Jul 13, 2024
  
  fb6af8bc
13 Jul, 2024 1 commit
- [ Misc ] More Cleanup of Marlin (#6359) · babf52da
  Robert Shaw authored Jul 13, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  babf52da
12 Jul, 2024 4 commits
- [Bugfix] Fix illegal memory access in FP8 MoE kernel (#6382) · 75f64d8b
  Cody Yu authored Jul 12, 2024
  
  75f64d8b
- [Bugfix] Fix dtype mismatch in PaliGemma (#6367) · 024ad87c
  Cyrus Leung authored Jul 12, 2024
  
  024ad87c
- [ Misc ] Support Models With Bias in `compressed-tensors` integration (#6356) · aea19f09
  Robert Shaw authored Jul 12, 2024
  
  aea19f09
- [ Misc ] Remove separate bias add (#6353) · 6047187c
  Robert Shaw authored Jul 12, 2024
  
  6047187c
11 Jul, 2024 4 commits
- [bug fix] Fix llava next feature size calculation. (#6339) · 1df43de9
  xwjiang2010 authored Jul 11, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
```
  1df43de9
- [ Misc ] Refactor Marlin Python Utilities (#6082) · b675069d
  Robert Shaw authored Jul 11, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  b675069d
- [Bugfix] GPTBigCodeForCausalLM: Remove lm_head from supported_lora_modules. (#6326) · 8a1415cf
  Thomas Parnell authored Jul 11, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
Co-authored-by: Travis Johnson <tsjohnso@us.ibm.com>
```
  8a1415cf
- [Doc] Remove comments incorrectly copied from another project (#6286) · 99ded1e1
  daquexian authored Jul 11, 2024
  
  99ded1e1
10 Jul, 2024 3 commits
- [Bugfix] MLPSpeculator: Use ParallelLMHead in tie_weights=False case. (#6303) · c38eba30
  Thomas Parnell authored Jul 10, 2024
```
Signed-off-by: Thomas Parnell <tpa@zurich.ibm.com>
```
  c38eba30
- [Bugfix] Support 2D input shape in MoE layer (#6287) · e72ae80b
  Woosuk Kwon authored Jul 10, 2024
  
  e72ae80b
- [Speculative Decoding] Medusa Implementation with Top-1 proposer (#4978) · 2416b26e
  Abhinav Goyal authored Jul 10, 2024
  
  2416b26e
09 Jul, 2024 1 commit
- [Bugfix]fix and needs_scalar_to_array logic check (#6238) · d3a24513
  Baoyuan Qi authored Jul 10, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
```
  d3a24513
08 Jul, 2024 3 commits
- [Bugfix] Mamba cache Cuda Graph padding (#6214) · ddc369fb
  tomeras91 authored Jul 08, 2024
  
  ddc369fb
- [Bugfix] use diskcache in outlines _get_guide #5436 (#6203) · 185ad31f
  Eric authored Jul 09, 2024
  
  185ad31f
- [Kernel] reloading fused_moe config on the last chunk (#6210) · f7a8fa39
  Avshalom Manevich authored Jul 08, 2024
  
  f7a8fa39
07 Jul, 2024 2 commits
- [ Misc ] Support Fp8 via `llm-compressor` (#6110) · abfe705a
  Robert Shaw authored Jul 07, 2024
```
Co-authored-by: Robert Shaw <rshaw@neuralmagic>
```
  abfe705a
- [Model] Add PaliGemma (#5189) · 6206dcb2
  Roger Wang authored Jul 06, 2024
```
Co-authored-by: Woosuk Kwon <woosuk.kwon@berkeley.edu>
```
  6206dcb2
05 Jul, 2024 2 commits
- [VLM] Cleanup validation and update docs (#6149) · ea4b5704
  Cyrus Leung authored Jul 05, 2024
  
  ea4b5704
- [VLM] Improve consistency between feature size calculation and dummy data for profiling (#6146) · a41357e9
  Roger Wang authored Jul 04, 2024
  
  a41357e9
04 Jul, 2024 2 commits
- [VLM] Calculate maximum number of multi-modal tokens by model (#6121) · ae96ef8f
  Cyrus Leung authored Jul 05, 2024
  
  ae96ef8f
- [Kernel][Model] logits_soft_cap for Gemma2 with flashinfer (#6051) · 69ec3ca1
  Lily Liu authored Jul 04, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  69ec3ca1
03 Jul, 2024 6 commits
- [ Misc ] Clean Up `CompressedTensorsW8A8` (#6113) · 62963d12
  Robert Shaw authored Jul 03, 2024
  
  62963d12
- [vlm] Remove vision language config. (#6089) · d9e98f42
  xwjiang2010 authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  d9e98f42
- [Kernel] Expand FP8 support to Ampere GPUs using FP8 Marlin (#5975) · 47f0954a
  Michael Goin authored Jul 03, 2024
  
  47f0954a
- [Bugfix] Fix `compute_logits` in Jamba (#6093) · 7cd2ebb0
  Roger Wang authored Jul 03, 2024
  
  7cd2ebb0
- [Core] Dynamic image size support for VLMs (#5276) · 9831aec4
  Cyrus Leung authored Jul 03, 2024
```
Signed-off-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: Xiaowei Jiang <xwjiang2010@gmail.com>
Co-authored-by: ywang96 <ywang@roblox.com>
Co-authored-by: xwjiang2010 <87673679+xwjiang2010@users.noreply.github.com>
Co-authored-by: Roger Wang <136131678+ywang96@users.noreply.github.com>
```
  9831aec4
- [hardware][misc] introduce platform abstraction (#6080) · 482045ee
  youkaichao authored Jul 02, 2024
  
  482045ee
02 Jul, 2024 1 commit

[Model] Jamba support (#4115) · 9d6a8daa

Mor Zusman authored Jul 03, 2024


Signed-off-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>
Co-authored-by: Erez Schwartz <erezs@ai21.com>
Co-authored-by: Mor Zusman <morz@ai21.com>
Co-authored-by: tomeras91 <57313761+tomeras91@users.noreply.github.com>
Co-authored-by: Tomer Asida <tomera@ai21.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
Co-authored-by: Muralidhar Andoorveedu <muralidhar.andoorveedu@centml.ai>

9d6a8daa