February Flow

Data Components:

Code:
The-Stack-V2

CodeText:
SE, whatever we've scraped

WebText:
HQ DCLM

DATA MIXES

~ 85% Source Code
~ 10% CodeText
~ 5% Webtext

~ 85% The-stack-V2
~ 15% CodeText
~ 0% Webtext

~ 100% Source Code

Deepseek Coder

StarCoder 2

Arctic