CoderEval

• 🗂️Benchmark Name: CoderEval
• 📚Publisher: arxiv
• 🏠Author Affiliation: Peking University and Huawei Cloud
• 🔗URL: https://github.com/CoderEval/CoderEval
• Number of Instances: 230 functions from 43 Python projects and 230 methods from 10 Java projects
• Problem Description’s Natural Language: English
• Code Solution’s Programming Language: Python and Java
• Data Statistics
  • Test Case: ✅
  • Average Number of Test Cases: /
  • Average Number of Characters in Problem Description: /
  • Average Number of Lines in Problem Description: /
  • Average Number of Characters in Code Solution: /
  • Average Number of Lines in Code Solution: /
• Scenario: Pragmatic Code Generation