🗂️Benchmark Name: MCoNaLa 📚Publisher: Arxiv 🏠Author Affiliation: Carnegie Mellon University 🔗URL: https://github.com/zorazrw/multilingual-conala Number of Instances: 896 Problem Descri...
PolyCoder
📙Paper: A systematic evaluation of large language models of code 📚Publisher: MAPS 🏠Author Affiliation: Carnegie Mellon University 🔑Public: ✅ 🌐Architecture Encoder-Decoder ...
CodeContests
🗂️Benchmark Name: CodeContests 📚Publisher: Science 🏠Author Affiliation: DeepMind 🔗URL: https://github.com/deepmind/code_contests Number of Instances: 165 Problem Description’s Natural ...
AlphaCode
📙Paper: Competition-level code generation with AlphaCode 📚Publisher: Science 🏠Author Affiliation: Deepmind 🔑Public: ❌ 🌐Architecture Encoder-Decoder Decoder-Only ...
JuPyT5
📙Paper: Training and Evaluating a Jupyter Notebook Data Science Assistant 📚Publisher: arxiv 🏠Author Affiliation: Microsoft 🔑Public: ❌ 🌐Architecture Encoder-Decoder Decod...
DSP
🗂️Benchmark Name: DSP 📚Publisher: Arxiv 🏠Author Affiliation: Microsoft 🔗URL: https://github.com/microsoft/DataScienceProblems Number of Instances: 1,119 Problem Description’s Natural L...
LAMDA
📙Paper: LaMDA Language Models for Dialog Applications 📚Publisher: arxiv 🏠Author Affiliation: Google 🔑Public: ❌ 🌐Architecture Encoder-Decoder Decoder-Only 📏Mode...
JigsawDataset
🗂️Benchmark Name: JigsawDataset-PandasEval1#2 📚Publisher: ICSE 🏠Author Affiliation: Microsoft 🔗URL: https://github.com/microsoft/JigsawDataset Problem Description’s Natural Language: Eng...
CodeParrot
📙Paper: CodeParrot 📚Publisher: other 🏠Author Affiliation: huggingface 🔑Public: ✅ 🌐Architecture Encoder-Decoder Decoder-Only 📏Model Size 110M; 1.5B ...
GPT-CC
📙Paper: GPT Code Clippy The Open Source version of GitHub Copilot 📚Publisher: other 🏠Author Affiliation: CodedotAI 🔑Public: ✅ 🌐Architecture Encoder-Decoder Decoder-Only ...
Recently Updated