Commits · 89579a201f2c84b512f0e1006ac2ea0d979803ab · OpenDAS / vllm_cscc

"tests/vscode:/vscode.git/clone" did not exist on "b7050ca7df640326f53e89f518f3ee045dfbbdef"

06 May, 2024 1 commit
- [Bugfix] Fix `asyncio.Task` not being subscriptable (#4623) · 323f27b9
  Cyrus Leung authored May 07, 2024
  
  323f27b9
04 May, 2024 2 commits
- [Bugfix] Fix inappropriate content of model_name tag in Prometheus metrics (#3937) · 43029870
  DearPlanet authored May 05, 2024
  
  43029870
- [Misc][Refactor] Introduce ExecuteModelData (#4540) · bc8ad684
  Cody Yu authored May 03, 2024
  
  bc8ad684
03 May, 2024 4 commits
- [Speculative decoding] Support target-model logprobs (#4378) · ab502751
  Cade Daniel authored May 03, 2024
  
  ab502751
- [Bugfix] Allow "None" or "" to be passed to CLI for string args that default to None (#4586) · 7e65477e
  Michael Goin authored May 03, 2024
  
  7e65477e
- [Core][Model runner refactoring 1/N] Refactor attn metadata term (#4518) · 3521ba4f
  SangBin Cho authored May 04, 2024
  
  3521ba4f
- [Misc] remove chunk detected debug logs (#4571) · ce3f1eed
  DefTruth authored May 03, 2024
  
  ce3f1eed
02 May, 2024 1 commit
- [Misc] centralize all usage of environment variables (#4548) · 5b8a7c1c
  youkaichao authored May 02, 2024
  
  5b8a7c1c
01 May, 2024 4 commits
- [Bugfix][Core] Fix and refactor logging stats (#4336) · 3a922c1e
  Roy authored May 02, 2024
  
  3a922c1e
- [Speculative decoding] Add ngram prompt lookup decoding (#4237) · b38e42fb
  leiwen83 authored May 02, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  b38e42fb
- [Bugfix] Fix 307 Redirect for `/metrics` (#4523) · 4dc8026d
  Robert Shaw authored May 01, 2024
  
  4dc8026d
- [Misc][Typo] type annotation fix (#4495) · f458112e
  harrywu authored May 01, 2024
  
  f458112e
30 Apr, 2024 1 commit
- [BugFix] fix num_lookahead_slots missing in async executor (#4165) · 4bb53e2d
  leiwen83 authored May 01, 2024
```
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
```
  4bb53e2d
28 Apr, 2024 2 commits
- Add more Prometheus metrics (#2764) · bf480c53
  Ronen Schaffer authored Apr 29, 2024
```
Co-authored-by: Robert Shaw <114415538+robertgshaw2-neuralmagic@users.noreply.github.com>
Co-authored-by: Robert Shaw <rshaw@neuralmagic.com>
```
  bf480c53
- [Misc] fix typo in llm_engine init logging (#4428) · 9c7306ac
  DefTruth authored Apr 28, 2024
  
  9c7306ac
27 Apr, 2024 4 commits
- [BugFix] Fix `min_tokens` when `eos_token_id` is None (#4389) · 81661da7
  Nick Hill authored Apr 27, 2024
```
Co-authored-by: DefTruth <31974251+deftruth@users.noreply.github.com>
```
  81661da7
- [Bugfix][Core] Fix get decoding config from ray (#4335) · 7134303c
  Roy authored Apr 27, 2024
  
  7134303c
- [Kernel] Full Tensor Parallelism for LoRA Layers (#3524) · eefeb164
  Austin Veselka authored Apr 27, 2024
```
Co-authored-by: Antoni Baum <antoni.baum@protonmail.com>
```
  eefeb164
- [Misc] Fix logger format typo (#4396) · 87f545ba
  Roy authored Apr 27, 2024
  
  87f545ba
26 Apr, 2024 2 commits
- [Core] Refactoring sampler and support prompt logprob for chunked prefill (#4309) · 603ad848
  SangBin Cho authored Apr 26, 2024
  
  603ad848
- [CI] Disable non-lazy string operation on logging (#4326) · a88081bf
  SangBin Cho authored Apr 26, 2024
```
Co-authored-by: Danny Guinther <dguinther@neuralmagic.com>
```
  a88081bf
25 Apr, 2024 2 commits
- [Core] Add `shutdown()` method to `ExecutorBase` (#4349) · 15e7c675
  Nick Hill authored Apr 25, 2024
  
  15e7c675
- [Core] Move ray_utils.py from `engine` to `executor` package (#4347) · 479d69fa
  Nick Hill authored Apr 24, 2024
  
  479d69fa
23 Apr, 2024 2 commits
- [Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. (#3951) · 62b8aebc
  Cade Daniel authored Apr 23, 2024
  
  62b8aebc
- [Mypy] Part 3 fix typing for nested directories for most of directory (#4161) · 0ae11f78
  SangBin Cho authored Apr 23, 2024
  
  0ae11f78
22 Apr, 2024 1 commit
- [Frontend] Enable support for CPU backend in AsyncLLMEngine. (#3993) · 077f0a2e
  Tao He authored Apr 22, 2024
```
Signed-off-by: Tao He <sighingnow@gmail.com>
```
  077f0a2e
21 Apr, 2024 1 commit
- Make initialization of tokenizer and detokenizer optional (#3748) · a37d815b
  GeauxEric authored Apr 21, 2024
```
Co-authored-by: Yun Ding <yunding@nvidia.com>
Co-authored-by: Roger Wang <ywang@roblox.com>
```
  a37d815b
20 Apr, 2024 2 commits
- Updating lm-format-enforcer version and adding links to decoding libraries in docs (#4222) · cc74b2b2
  Noam Gat authored Apr 20, 2024
  
  cc74b2b2
- Fix missing docs and out of sync `EngineArgs` (#4219) · 682789d4
  Harry Mellor authored Apr 20, 2024
```
Co-authored-by: Harry Mellor <hmellor@oxts.com>
```
  682789d4
19 Apr, 2024 2 commits
- [Bugfix][Core] Restore logging of stats in the async engine (#4150) · 7be4f562
  Ronen Schaffer authored Apr 19, 2024
  
  7be4f562
- Support eos_token_id from generation_config.json (#4182) · a134ef6f
  Simon Mo authored Apr 18, 2024
  
  a134ef6f
18 Apr, 2024 3 commits
- [CI/CD] add neuron docker and ci test scripts (#3571) · cd2f63fb
  Liangfu Chen authored Apr 18, 2024
  
  cd2f63fb
- [Bugfix] Get available quantization methods from quantization registry (#4098) · 53b018ed
  Michael Goin authored Apr 18, 2024
  
  53b018ed
- [Typing] Mypy typing part 2 (#4043) · 533d2a1f
  SangBin Cho authored Apr 18, 2024
```
Co-authored-by: SangBin Cho <sangcho@sangcho-LT93GQWG9C.local>
```
  533d2a1f
17 Apr, 2024 1 commit
- [Core] RayWorkerVllm --> WorkerWrapper to reduce duplication (#4024) · 8438e056
  youkaichao authored Apr 17, 2024
```
[Core] replace narrow-usage RayWorkerVllm to general WorkerWrapper to reduce code duplication (#4024)
```
  8438e056
16 Apr, 2024 4 commits
- [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894) · e95cd879
  Cade Daniel authored Apr 16, 2024
  
  e95cd879
- [Core] Refactor model loading code (#4097) · 69e1d2fb
  Antoni Baum authored Apr 16, 2024
  
  69e1d2fb
- LM Format Enforcer Guided Decoding Support (#3868) · 05434764
  Noam Gat authored Apr 16, 2024
```
Co-authored-by: Simon Mo <simon.mo@hey.com>
```
  05434764
- [Core] Fix engine-use-ray broken (#4105) · 4e7ee664
  SangBin Cho authored Apr 16, 2024
  
  4e7ee664
14 Apr, 2024 1 commit
- [Frontend] [Core] feat: Add model loading using `tensorizer` (#3476) · 711a0002
  Sanger Steel authored Apr 13, 2024
  
  711a0002