MTPB

• 🗂️Benchmark Name: MTPB
• 📚Publisher: arxiv
• 🏠Author Affiliation: Salesforce Research
• 🔗URL: https://github.com/salesforce/CodeGen/tree/main/benchmark
• Number of Instances: 115
• Problem Description’s Natural Language: English
• Code Solution’s Programming Language: Python
• Data Statistics
  • Test Case: ✅
  • Average Number of Test Cases: /
  • Average Number of Characters in Problem Description: 72.7
  • Average Number of Lines in Problem Description: 1.0
  • Average Number of Characters in Code Solution: /
  • Average Number of Lines in Code Solution: /
• Scenario: Multi-Turn