Commits · 7013a80170146ca4f7fd9539e4d7fd852d4718b6 · kecinstone / 2024pra-vllm

30 Oct, 2023 1 commit
- Add support for `spaces_between_special_tokens` · 7013a801
  Dan Lord authored Oct 30, 2023
  
  7013a801
29 Oct, 2023 1 commit
- fix: don't skip first special token. (#1497) · beac8dd4
  Ricardo Lu authored Oct 29, 2023
  
  beac8dd4
13 Oct, 2023 3 commits
- Fix the issue for AquilaChat2-* models (#1339) · de894728
  Lu Wang authored Oct 13, 2023
  
  de894728
- Bump up transformers version & Remove MistralConfig (#1254) · e7c8555d
  Woosuk Kwon authored Oct 13, 2023
  
  e7c8555d
- Improve detokenization performance (#1338) · ec3b5ce9
  Antoni Baum authored Oct 13, 2023
  
  ec3b5ce9
03 Oct, 2023 1 commit
- add support for tokenizer revision (#1163) · 66d18a7f
  Federico Cassano authored Oct 02, 2023
```
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  66d18a7f
28 Sep, 2023 2 commits
- Fix Mistral model (#1220) · a8e98aee
  Woosuk Kwon authored Sep 28, 2023
  
  a8e98aee
- [Mistral] Mistral-7B-v0.1 support (#1196) · bb1ba58f
  Chris Bamford authored Sep 28, 2023
```
Co-authored-by: timlacroix <t@mistral.ai>
```
  bb1ba58f
27 Sep, 2023 1 commit
- fix qwen-14b model (#1173) · 28e616c4
  Qing authored Sep 28, 2023
  
  28e616c4
15 Sep, 2023 1 commit
- Fix warning message on LLaMA FastTokenizer (#1037) · 64ca424e
  Woosuk Kwon authored Sep 14, 2023
  
  64ca424e
14 Sep, 2023 1 commit
- Fix detokenization leaving special tokens (#1044) · dd54a4b0
  Antoni Baum authored Sep 14, 2023
```
Signed-off-by: Antoni Baum <antoni.baum@protonmail.com>
```
  dd54a4b0
13 Sep, 2023 2 commits
- Add Model Revision Support (#1014) · ab019eea
  Jasmond L authored Sep 14, 2023
```
Co-authored-by: Jasmond Loh <Jasmond.Loh@hotmail.com>
Co-authored-by: Zhuohan Li <zhuohan123@gmail.com>
```
  ab019eea
- Use TGI-like incremental detokenization (#984) · 9841d48a
  Antoni Baum authored Sep 13, 2023
  
  9841d48a
04 Sep, 2023 1 commit
- Only emit warning about internal tokenizer if it isn't being used (#939) · e15932bb
  Nelson Liu authored Sep 04, 2023
  
  e15932bb
22 Aug, 2023 1 commit

Add support for aquila (#663) · ad5f2fe3

shunxing1234 authored Aug 22, 2023



* add aquila
Signed-off-by: ftgreat <ftgreat@163.com>

* fix some bug
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete pdb
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* fix bugs
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* delete whitespace
Signed-off-by: shunxing1234 <xw747777271@gmail.com>

* format

* fix order

---------
Signed-off-by: ftgreat <ftgreat@163.com>
Signed-off-by: shunxing1234 <xw747777271@gmail.com>
Co-authored-by: ftgreat <ftgreat@163.com>

ad5f2fe3

15 Aug, 2023 1 commit
- Fix typo in tokenizer.py (#750) · 805de738
  Ikko Eltociear Ashimine authored Aug 15, 2023
```
conjuction -> conjunction
```
  805de738
08 Aug, 2023 1 commit
- add QWen-7b (#685) · a57d13cc
  Qing authored Aug 09, 2023
```
Co-authored-by: wq.chu <wq.chu@tianrang-inc.com>
```
  a57d13cc
02 Aug, 2023 1 commit
- Add Falcon support (new) (#592) · 1b0bd0fe
  Zhuohan Li authored Aug 02, 2023
  
  1b0bd0fe
17 Jul, 2023 1 commit
- Add support for baichuan (#365) · 20b0d88d
  codethazine authored Jul 17, 2023
  
  20b0d88d
12 Jul, 2023 1 commit
- Fix handling of special tokens in decoding. (#418) · c6dfc3cd
  xcnick authored Jul 12, 2023
  
  c6dfc3cd
08 Jul, 2023 1 commit
- Add trust_remote_code arg to get_config (#405) · ddfdf470
  Woosuk Kwon authored Jul 08, 2023
  
  ddfdf470
07 Jul, 2023 1 commit
- Add trust-remote-code flag to handle remote tokenizers (#364) · a945fcc2
  codethazine authored Jul 07, 2023
  
  a945fcc2
03 Jul, 2023 2 commits
- [Model] Add support for MPT (#334) · 404422f4
  Woosuk Kwon authored Jul 03, 2023
  
  404422f4
- [Quality] Add code formatter and linter (#326) · d6fa1be3
  Zhuohan Li authored Jul 03, 2023
  
  d6fa1be3
28 Jun, 2023 2 commits
- [Tokenizer] Add tokenizer mode (#298) · 998d9d15
  Woosuk Kwon authored Jun 28, 2023
  
  998d9d15
- [Tokenizer] Add an option to specify tokenizer (#284) · 4338cc47
  Woosuk Kwon authored Jun 28, 2023
  
  4338cc47