1. 04 Aug, 2025 1 commit
    • Idan Tene's avatar
      Fix humaneval_instruct (#3201) · edf3aa7a
      Idan Tene authored
      * Update humaneval_64_instruct.yaml
      
      Sync doc_to_text with humaneval_instruct.yaml
      
      * Update humaneval_instruct.yaml
      
      Remove redundant (flawed) spaces
      
      * Update README.md
      
      * Bump task version
      edf3aa7a
  2. 03 Jul, 2025 1 commit
  3. 30 Jun, 2025 1 commit
    • jinze's avatar
      FixBug: Align the Humaneval with official results for Llama-3.1-70B-Instruct (#3092) · a7ca0435
      jinze authored
      * Fix: Align the Humaneval dataset with official results
      
      Details:(1) modified the "doc_to_text" and "gen_prefix" in the "humaneval_instruct.yaml" file to make them the same as the Prompt in "meta-llama/Llama-3.1-70B-Instruct-evals".
      
      (2) Change r.rfind("```") to r.find("```"), so it can locate the first "```", not the last one.
      
      Results: Partially reproduced the official results: The result of LLaMA3.1-8B-Instruct is 66.5 (the official result is 72.6), and the result of LLaMA3.1-70B-Instruct is 80.5 (the official result is 80.5).
      
      Ref: PR#2650
      
      * add changelog and version
      
      * add changelog
      a7ca0435
  4. 20 Mar, 2025 1 commit
  5. 11 Mar, 2025 1 commit
  6. 25 Feb, 2025 1 commit
  7. 15 Jan, 2025 1 commit