Abstract

This paper addresses the increasing demand for accurate, fair, and efficient grading of essay-style assessments in higher education by integrating institutional requirements with recent advances in large language models (LLMs). We propose a pipeline emphasizing privacy, explainability, consistency, and fairness. To ensure privacy, the system operates on local servers and employs rigorous anonymization of student data. Grading events integrate task prompts, instructor guidelines, student submissions, grader commentary, and final scores into structured records, enhancing evaluation accuracy and transparency. We detail the development and validation process, fine-tuning two local LLMs using historical course data. Results demonstrate the models’ ability to effectively replicate original grading decisions while safeguarding student data. Additionally, we discuss how this framework aligns with the interests of students, educators, and policymakers. Our approach establishes a foundational methodology for responsibly integrating AI-driven grading into higher education. This contribution fosters trust among stakeholders and sets a clear direction for future implementation and research efforts.

Arz von Straussenburg, A. F., Wolters, A., Aldenhoff, T. T., & Riehle, D. M. (2025). Enabling Responsible LLM-Based Grading in Higher Education – Design Guidelines and a Reproducible Data Preparation Pipeline. In S. Chatterjee, J. Brocke, & R. Anderson (Eds.), Local Solutions for Global Challenges : 20th International Conference on Design Science Research in Information Systems and Technology, DESRIST 2025, Montego Bay, Jamaica, June 2-4, 2025, Proceedings, Part II (pp. 3–20). Springer Nature Switzerland. https://doi.org/10.1007/978-3-031-93979-2_1

@inproceedings{Arz2025Enabling,
	address = {Cham},
	author = {Arz von Straussenburg, Arnold F. and Wolters, Anna and Aldenhoff, Timon T. and Riehle, Dennis M.},
	booktitle = {Local {Solutions} for {Global} {Challenges} : 20th {International} {Conference} on {Design} {Science} {Research} in {Information} {Systems} and {Technology}, {DESRIST} 2025, {Montego} {Bay}, {Jamaica}, {June} 2-4, 2025, {Proceedings}, {Part} {II}},
	doi = {10.1007/978-3-031-93979-2_1},
	editor = {Chatterjee, Samir and Brocke, Jan and Anderson, Ricardo},
	year = {2025},
	pages = {3--20},
	organization = {Springer Nature Switzerland},
	title = {Enabling {Responsible} {LLM}-{Based} {Grading} in {Higher} {Education} -- {Design} {Guidelines} and a {Reproducible} {Data} {Preparation} {Pipeline}},
	url = {https://doi.org/10.1007/978-3-031-93979-2_1},
}