Skip to content

GitLab

"vscode:/vscode.git/clone" did not exist on "712216928fa252d6592a1518579018a69cb72bfe"
Switch branch/tag
Find file BlameHistoryPermalink
  • Nicolas Patry's avatar
    We can have a tokenizer anywhere. (#2527) · 90b226db
    Nicolas Patry authored 
    * We can have a tokenizer anywhere.

* Handling potential lack of offsets (python tokenizer)

* Remove redundancy.

* Fixing the tests.

* Flake.lock update ?

* Fixing the  GIL locking.

* Fixing mamba by using the transformers version.

* Adding the legacy handle.

* Ellide lifetime.

* Lint.

* Deprecation message.

* Fixing bad rebase.
    90b226db
__init__.py 50 KB