-
wang.yuqi authored
[Bugfix] The special_tokens in tokenizer should also be controlled by do_lower_case in encoder_config. (#20750) Signed-off-by:wang.yuqi <noooop@126.com>
5895afd7
[Bugfix] The special_tokens in tokenizer should also be controlled by do_lower_case in encoder_config. (#20750)
Signed-off-by: wang.yuqi <noooop@126.com>