Commits · 211318d44ad3225eaba488150f797efceab6cc32 · norm / vllm

28 May, 2023 2 commits
- Add throughput benchmarking script (#133) · 211318d4
  Woosuk Kwon authored May 28, 2023
  
  211318d4
- Enable LLaMA fast tokenizer (#132) · 337871c6
  Woosuk Kwon authored May 28, 2023
  
  337871c6
27 May, 2023 2 commits
- Add a doc for installation (#128) · 56b7f0ef
  Woosuk Kwon authored May 27, 2023
  
  56b7f0ef
- Improve setup script & Add a guard for bfloat16 kernels (#130) · d7211684
  Woosuk Kwon authored May 27, 2023
  
  d7211684
25 May, 2023 1 commit
- Add activation registry (#126) · 4a151dd4
  Woosuk Kwon authored May 25, 2023
  
  4a151dd4
24 May, 2023 4 commits
- OpenAI Compatible Frontend (#116) · 057daef7
  Zhuohan Li authored May 23, 2023
  
  057daef7
- Incrementally decode output tokens (#121) · e8671783
  Woosuk Kwon authored May 23, 2023
  
  e8671783
- Print warnings/errors for large swap space (#123) · aedba6d5
  Woosuk Kwon authored May 23, 2023
  
  aedba6d5
- Add contributing guideline and mypy config (#122) · a283ec2e
  Woosuk Kwon authored May 23, 2023
  
  a283ec2e
23 May, 2023 2 commits
- Fix latency benchmark script (#118) · 3f942acf
  Woosuk Kwon authored May 22, 2023
  
  3f942acf
- Add initial sphinx docs (#120) · 19d28994
  Woosuk Kwon authored May 22, 2023
  
  19d28994
22 May, 2023 1 commit
- Introduce LLM class for offline inference (#115) · 655a5e48
  Woosuk Kwon authored May 21, 2023
  
  655a5e48
21 May, 2023 1 commit
- Implement stop strings and best_of (#114) · f746ced0
  Woosuk Kwon authored May 21, 2023
  
  f746ced0
20 May, 2023 2 commits
- Refactor system architecture (#109) · c3442c1f
  Woosuk Kwon authored May 20, 2023
  
  c3442c1f
- Remove unused parts in Megatron-LM code and add copyright notice (#110) · 7297fa6f
  Zhuohan Li authored May 20, 2023
  
  7297fa6f
19 May, 2023 2 commits
- Fix timeout error in the FastAPI frontend (#34) · b7955ef1
  Zhuohan Li authored May 19, 2023
  
  b7955ef1
- Use runtime profiling to replace manual memory analyzers (#81) · f756799b
  Zhuohan Li authored May 19, 2023
  
  f756799b
18 May, 2023 1 commit
- Use pytest format for unit tests (#107) · 825d8892
  Woosuk Kwon authored May 17, 2023
  
  825d8892
15 May, 2023 4 commits
- Add docstrings to some modules and classes (#100) · b322fd16
  Woosuk Kwon authored May 14, 2023
  
  b322fd16
- Add copyright headers to source files adapted from FT (#104) · 667ba399
  Woosuk Kwon authored May 14, 2023
  
  667ba399
- Add copyright headers for HF models (#103) · 707ec647
  Woosuk Kwon authored May 14, 2023
  
  707ec647
- Add Apache-2.0 license (#102) · 89988ec8
  Woosuk Kwon authored May 14, 2023
  
  89988ec8
13 May, 2023 1 commit
- Minor code cleaning for SamplingParams (#99) · 6208d622
  Woosuk Kwon authored May 12, 2023
  
  6208d622
11 May, 2023 2 commits
- Enhance SamplingParams (#96) · 42f1042e
  Woosuk Kwon authored May 11, 2023
  
  42f1042e
- Implement presence and frequency penalties (#95) · 55f8b0a5
  Woosuk Kwon authored May 10, 2023
  
  55f8b0a5
10 May, 2023 4 commits
- Support top-k sampling (#94) · 9f88db35
  Woosuk Kwon authored May 10, 2023
  
  9f88db35
- Avoid sorting waiting queue & Minor code cleaning (#93) · ae356774
  Woosuk Kwon authored May 10, 2023
  
  ae356774
- Log system stats (#90) · e3319577
  Woosuk Kwon authored May 10, 2023
  
  e3319577
- Rename variables and methods (#91) · 8d66a7b6
  Woosuk Kwon authored May 10, 2023
  
  8d66a7b6
09 May, 2023 5 commits
- Update sample prompts in `simple_server.py` (#89) · ce26e57f
  Woosuk Kwon authored May 09, 2023
  
  ce26e57f
- Use slow tokenizer for LLaMA (#84) · 85eb6318
  Woosuk Kwon authored May 09, 2023
  
  85eb6318
- Enhance model loader (#83) · add055e1
  Woosuk Kwon authored May 09, 2023
  
  add055e1
- Refactor system architecture (#82) · 7c041ab5
  Woosuk Kwon authored May 09, 2023
  
  7c041ab5
- Add a system logger (#85) · 8917782a
  Woosuk Kwon authored May 08, 2023
  
  8917782a
07 May, 2023 1 commit
- Specify python package dependencies in requirements.txt (#78) · 7addca59
  Woosuk Kwon authored May 07, 2023
  
  7addca59
06 May, 2023 1 commit
- [Minor] Fix a dtype bug (#79) · c84e9242
  Woosuk Kwon authored May 06, 2023
  
  c84e9242
05 May, 2023 1 commit
- Replace FlashAttention with xformers (#70) · c9d5b6d4
  Woosuk Kwon authored May 05, 2023
  
  c9d5b6d4
04 May, 2023 3 commits
- Use dtype from model config & Add Dolly V2 (#63) · 189ae231
  Woosuk Kwon authored May 04, 2023
  
  189ae231
- Add support for GPT-2 (#60) · e548c148
  Woosuk Kwon authored May 04, 2023
  
  e548c148
- Fix a bug in attention kernel (#68) · 130d5fd8
  Woosuk Kwon authored May 04, 2023
  
  130d5fd8