V-February Flow

Data Components:

Code:
The-Stack-V2

CodeText:
SE, whatever we've scraped

WebText:
HQ DCLM

DATA MIXES

~85% Source Code
~10% CodeText
~5% Webtext

~85% The-stack-V2
~15% CodeText
~0% Webtext

~100% Source Code

Deepseek Coder

StarCoder 2

Arctic