CSEPrompts：计算机科学基础教学提示的评估标准

发布时间：2024年04月03日

LLM应用

计算机科学

CSEPrompts: A Benchmark of Introductory Computer Science Prompts

摘要

AI、机器学习和NLP的最新突破催生了新一代的大型语言模型（LLMs），这些模型训练于海量数据，参数规模高达万亿级别。随着商业应用如ChatGPT的推广，公众得以接触并利用LLMs来创作优质的学术和专业文本。教育机构发现学生日益依赖AI生成的内容，因此正积极研究这项技术的影响和可能的不当使用。特别是在计算机科学（CS）及其相关领域，LLMs还能编写多种编程语言的代码，这对教育课程产生了显著影响。为了深入探讨LLMs在CS教育中的作用，我们推出了CSEPrompts框架，汇集了众多来自初级CS和编程课程的编程练习和选择题。此外，我们还展示了实验数据，用以评估不同LLMs在编写Python代码和解答计算机科学基础问题上的表现。

Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs) that are trained on massive amounts of data and often have trillions of parameters. Commercial applications (e.g., ChatGPT) have made this technology available to the general public, thus making it possible to use LLMs to produce high-quality texts for academic and professional purposes. Schools and universities are aware of the increasing use of AI-generated content by students and they have been researching the impact of this new technology and its potential misuse. Educational programs in Computer Science (CS) and related fields are particularly affected because LLMs are also capable of generating programming code in various programming languages. To help understand the potential impact of publicly available LLMs in CS education, we introduce CSEPrompts, a framework with hundreds of programming exercise prompts and multiple-choice questions retrieved from introductory CS and programming courses. We also provide experimental results on CSEPrompts to evaluate the performance of several LLMs with respect to generating Python code and answering basic computer science and programming questions.

Arxiv

摘要

AI、机器学习和NLP的最新突破催生了新一代的大型语言模型（LLMs），这些模型训练于海量数据，参数规模高达万亿级别。随着商业应用如ChatGPT的推广，公众得以接触并利用LLMs来创作优质的学术和专业文本。教育机构发现学生日益依赖AI生成的内容，因此正积极研究这项技术的影响和可能的不当使用。特别是在计算机科学（CS）及其相关领域，LLMs还能编写多种编程语言的代码，这对教育课程产生了显著影响。为了深入探讨LLMs在CS教育中的作用，我们推出了CSEPrompts框架，汇集了众多来自初级CS和编程课程的编程练习和选择题。此外，我们还展示了实验数据，用以评估不同LLMs在编写Python代码和解答计算机科学基础问题上的表现。

Recent advances in AI, machine learning, and NLP have led to the development of a new generation of Large Language Models (LLMs) that are trained on massive amounts of data and often have trillions of parameters. Commercial applications (e.g., ChatGPT) have made this technology available to the general public, thus making it possible to use LLMs to produce high-quality texts for academic and professional purposes. Schools and universities are aware of the increasing use of AI-generated content by students and they have been researching the impact of this new technology and its potential misuse. Educational programs in Computer Science (CS) and related fields are particularly affected because LLMs are also capable of generating programming code in various programming languages. To help understand the potential impact of publicly available LLMs in CS education, we introduce CSEPrompts, a framework with hundreds of programming exercise prompts and multiple-choice questions retrieved from introductory CS and programming courses. We also provide experimental results on CSEPrompts to evaluate the performance of several LLMs with respect to generating Python code and answering basic computer science and programming questions.

CSEPrompts：计算机科学基础教学提示的评估标准

CSEPrompts：计算机科学基础教学提示的评估标准

摘要

Related Skills

Nano Banana Pro

Markdown Converter

1password

CSEPrompts：计算机科学基础教学提示的评估标准

摘要