Skip to content

GitLab

"...git@developer.sourcefind.cn:OpenDAS/megatron-lm.git" did not exist on "688448db7547be90203440cfd105703d8a853f39"
Unverified Commit 3ebe17eb authored by Atream's avatar Atream Committed by GitHub
Browse files

Merge pull request #699 from kvcache-ai/Atream-patch-1 

Update DeepseekR1_V3_tutorial.md
parents 9650893a 369f4d91
Show whitespace changes
Inline Side-by-side
Showing with 1 addition and 1 deletion
doc/en/DeepseekR1_V3_tutorial.md
View file @ 3ebe17eb
...@@ -160,7 +160,7 @@ is speed up which is inspiring. So our showcase makes use of this finding* ...@@ -160,7 +160,7 @@ is speed up which is inspiring. So our showcase makes use of this finding*
### V0.2.2 longer context & FP8 kernel ### V0.2.2 longer context & FP8 kernel
#### longer context #### longer context
To use this feature, [install flashinfer](https://github.com/flashinfer-ai/flashinfer) first. To use this feature, [install flashinfer](https://github.com/flashinfer-ai/flashinfer) first.
Note: The latest MLA kernel in FlashInfer still has a few minor issues. They are continuously fixing them on the main branch. If you are using FlashInfer, please install it from the main source code.
If you want to use long context(longer than 20K) for prefill, enable the matrix absorption MLA during the prefill phase, which will significantly reduce the size of the kv cache. Modify yaml file like this: If you want to use long context(longer than 20K) for prefill, enable the matrix absorption MLA during the prefill phase, which will significantly reduce the size of the kv cache. Modify yaml file like this:
``` ```
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment