• Wis Kojohnjaratkul's avatar
    [WIP] Add IFEval / Instruction-Following Eval (#1087) · aa61f940
    Wis Kojohnjaratkul authored
    * Add IFEval task
    
    * Check and download nltk punkt if not already downloaded
    
    * Update gen_max_toks to 2048 to support "900 words+" instructions
    
    * Resolve pre-commit linting issues
    
    * Reduce max_gen_toks to 1280 to conserve token usage
    
    * Add warning message in `process_results` call for non chat-finetuned models
    aa61f940
pyproject.toml 2.28 KB