Commits · 93872128111499f2d9127c3cd1f94b35850187d1 · OpenDAS / vllm_cscc

"tests/models/test_big_models.py" did not exist on "f342153b4892789616a9bf58b6b9348dcb2329c3"

26 Sep, 2024 1 commit
- support head_dim 160 and update benchmark_throughput.py · 93872128
  zhuwenwen authored Sep 26, 2024
  
  93872128
25 Sep, 2024 1 commit
- remove reason of unsupported · a087cda8
  zhuwenwen authored Sep 25, 2024
  
  a087cda8
24 Sep, 2024 1 commit
- 优化pa小batch性能（pa_v2），优化pa小seq性能(pa_v1)，reusekv=16优化 · 2b91ac93
  zhuwenwen authored Sep 24, 2024
  
  2b91ac93
20 Sep, 2024 5 commits
- [Core] Change multiproc method default to spawn · de7d9456
  zhuwenwen authored Sep 20, 2024
  
  de7d9456
- add pa tc · 0c70376b
  zhuwenwen authored Sep 20, 2024
  
  0c70376b
- default setting FA_PAD=1 · fe1ec8c5
  zhuwenwen authored Sep 20, 2024
  
  fe1ec8c5
- choose ck or cutlass implementation based on the fa version · 7fafc730
  zhuwenwen authored Sep 20, 2024
  
  7fafc730
- add benchmarks to vllm whl · 5b62725e
  zhuwenwen authored Sep 20, 2024
  
  5b62725e
19 Sep, 2024 1 commit
- optimize deepseek_v2_lite fused_moe kernel config · 77a13b1b
  zhuwenwen authored Sep 19, 2024
  
  77a13b1b
18 Sep, 2024 1 commit
- update platforms init · bc84829b
  zhuwenwen authored Sep 18, 2024
  
  bc84829b
13 Sep, 2024 11 commits
- bump version to v0.6.1.post2 (#8473) · 9ba0817f
  Simon Mo authored Sep 13, 2024
  
  9ba0817f
- [HotFix] Fix final output truncation with stop string + streaming (#8468) · 18e9e1f7
  Nick Hill authored Sep 13, 2024
  
  18e9e1f7
- [plugin][torch.compile] allow to add custom compile backend (#8445) · 0a4806f0
  youkaichao authored Sep 13, 2024
  
  0a4806f0
- remove the automatic switching strategy of fa · 408f663a
  zhuwenwen authored Sep 13, 2024
  
  408f663a
- update torch version and support lm_head nn layout · aa1e273a
  zhuwenwen authored Sep 13, 2024
  
  aa1e273a
- [Misc] Skip loading extra bias for Qwen2-VL GPTQ-Int8 (#8442) · 06311e29
  Jee Jee Li authored Sep 13, 2024
  
  06311e29
- bump version to v0.6.1.post1 (#8440) · acda0b35
  Simon Mo authored Sep 12, 2024
  
  acda0b35
- [bugfix] torch profiler bug for single gpu with GPUExecutor (#8354) · ba775279
  William Lin authored Sep 12, 2024
  
  ba775279
- [Bugfix] Fix async log stats (#8417) · 68210201
  Alexander Matveev authored Sep 12, 2024
  
  68210201
- [Core] Factor out input preprocessing to a separate class (#7329) · 5ec9c0fb
  Cyrus Leung authored Sep 13, 2024
  
  5ec9c0fb
- [BugFix] fix group_topk (#8430) · 8f44a92d
  Dipika Sikka authored Sep 12, 2024
  
  8f44a92d
12 Sep, 2024 19 commits
- [Bugfix] Fix weight loading issue by rename variable. (#8293) · a480939e
  Wenxiang authored Sep 13, 2024
  
  a480939e
- [Hotfix][Pixtral] Fix multiple images bugs (#8415) · d31174a4
  Patrick von Platen authored Sep 13, 2024
  
  d31174a4
- [Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for... · c1636945
  Roger Wang authored Sep 12, 2024
```
[Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for multimodal models (#8425)
```
  c1636945
- [Bugfix] multi-step + flashinfer: ensure cuda graph compatible (#8427) · 01987725
  Alexander Matveev authored Sep 12, 2024
  
  01987725
- [Core] Add engine option to return only deltas or final output (#7381) · 551ce010
  Nick Hill authored Sep 12, 2024
  
  551ce010
- [multi-step] add flashinfer backend (#7928) · a6c0f365
  William Lin authored Sep 12, 2024
  
  a6c0f365
- [Bugfix] Offline mode fix (#8376) · f2e263b8
  Joe Runde authored Sep 12, 2024
```
Signed-off-by: Joe Runde <Joseph.Runde@ibm.com>
```
  f2e263b8
- [BugFix] Fix Duplicate Assignment in Hermes2ProToolParser (#8423) · 1f0c75af
  Luis Vega authored Sep 12, 2024
  
  1f0c75af
- [BugFix] lazy init _copy_stream to avoid torch init wrong gpu instance (#8403) · 8a23e933
  WANGWEI authored Sep 13, 2024
  
  8a23e933
- [Model] Support multiple images for qwen-vl (#8247) · c6202dae
  Alex Brooks authored Sep 12, 2024
```
Signed-off-by: Alex-Brooks <Alex.Brooks@ibm.com>
Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com>
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  c6202dae
- [Bugfix] Fix InternVL2 inference with various num_patches (#8375) · e56bf277
  Isotr0py authored Sep 13, 2024
```
Co-authored-by: DarkLight1337 <tlleungac@connect.ust.hk>
```
  e56bf277
- [Hotfix][VLM] Fixing max position embeddings for Pixtral (#8399) · 520ca380
  Roger Wang authored Sep 12, 2024
  
  520ca380
- [torch.compile] hide slicing under custom op for inductor (#8384) · 7de49aa8
  youkaichao authored Sep 12, 2024
  
  7de49aa8
- [Misc] Use RoPE cache for MRoPE (#8396) · 42ffba11
  Woosuk Kwon authored Sep 11, 2024
  
  42ffba11
- [Misc] Raise error when using encoder/decoder model with cpu backend (#8355) · 295c4730
  Kevin Lin authored Sep 12, 2024
  
  295c4730
- [Gemma2] add bitsandbytes support for Gemma2 (#8338) · 1bf2dd9d
  Blueyo0 authored Sep 12, 2024
  
  1bf2dd9d
- [Bugfix]: Fix the logic for deciding if tool parsing is used (#8366) · 5a60699c
  tomeras91 authored Sep 12, 2024
  
  5a60699c
- Fix the AMD weight loading tests (#8390) · b6c75e1c
  Michael Goin authored Sep 11, 2024
  
  b6c75e1c
- [TPU] Use Ray for default distributed backend (#8389) · b71c956d
  Woosuk Kwon authored Sep 11, 2024
  
  b71c956d