Unverified Commit 06b05d45 authored by Luc CAILLIAU's avatar Luc CAILLIAU Committed by GitHub
Browse files

Clarify stride option (#22684)

* Clarify stride option

* formatting
parent 0224aaf6
...@@ -68,7 +68,8 @@ class AggregationStrategy(ExplicitEnum): ...@@ -68,7 +68,8 @@ class AggregationStrategy(ExplicitEnum):
same entity together in the predictions or not. same entity together in the predictions or not.
stride (`int`, *optional*): stride (`int`, *optional*):
If stride is provided, the pipeline is applied on all the text. The text is split into chunks of size If stride is provided, the pipeline is applied on all the text. The text is split into chunks of size
model_max_length. Works only with fast tokenizers and `aggregation_strategy` different from `NONE`. model_max_length. Works only with fast tokenizers and `aggregation_strategy` different from `NONE`. The
value of this argument defines the number of overlapping tokens between chunks.
aggregation_strategy (`str`, *optional*, defaults to `"none"`): aggregation_strategy (`str`, *optional*, defaults to `"none"`):
The strategy to fuse (or not) tokens based on the model prediction. The strategy to fuse (or not) tokens based on the model prediction.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment