Skip to content

GitLab

Switch branch/tag
  1. 11 Apr, 2022 1 commit
    • SaulLu's avatar
      add a warning in `SpmConverter` for sentencepiece's model using the byte fallback feature (#16629) · 1025a9b7
      SaulLu authored 
      * update proto sentencepiece model

* Revert "update proto sentencepiece model"

This reverts commit b07f671747fec35773d0b3d4788b8b15aefa0229.

* add check

* add test

* Revert "Revert "update proto sentencepiece model""

This reverts commit 46108257b8927b73627ec8f4f3eed53a95fc700d.

* test for log level

* test for log level 2

* warning at the warning level

* clean

* format

* add explanation in docstring
      1025a9b7