V-February Flow

Data Components:
- Code: The-Stack-V2
- CodeText: SE, whatever we’ve scraped
- WebText: HQ DCLM

DATA MIXES
- ~85% Source Code
- ~10% CodeText
- ~5% Webtext

- ~85% The-stack-V2
- ~15% CodeText
- ~0% Webtext

- ~100% Source Code

Deepseek Coder

StarCoder 2

Arctic