Automated Assessment in Higher Education

1. Automated Assessment in Higher Education

This learning material introduces contemporary technologies and methods of automated assessment in higher education. The emphasis is on analysing systems such as:

• automated quizzes
• essay checking software
• portfolio analysis tools
• the application of artificial intelligence in assessment

Two case studies involving automated assessment are also presented. This material enables you to explore the advantages and challenges of automation in assessment and encourages reflection on the possibilities of applying these approaches in your own educational practice.

2. Introduction

Key questions include:

• Can technology replace or enhance teacher-led assessment?
• In which situations and in what ways can technology contribute to more efficient or more reliable assessment?

3. Examples of Technologies and Approaches

• Automated quizzes – most commonly include multiple-choice questions, true/false items, matching tasks, and numerical answers. This type of assessment provides immediate feedback and can be widely used in both formative and summative assessment, especially in large groups (Divjak et al., 2022; Clark, 2012).
• Essay analysis tools – software such as GrammarlyGO and tools based on large language models (LLMs), such as ChatGPT, are increasingly used for automatic analysis of grammar, coherence, argumentation, and style. These tools support self-assessment and provide personalised feedback to students (Kumar et al., 2024).
• Automated portfolio assessment – modern systems integrated into LMS environments use rubrics and checklists for structured evaluation of student work. This automates the allocation of points based on predefined criteria (e.g., evidence of reflection, quality of argumentation, clarity of presentation) (Khasawneh et al., 2025). For example, in the Mahara system or the Moodle Assignment activity, a student submits multiple artefacts (an essay, an infographic, a video), and the instructor selects a rubric that automatically assigns points depending on achievement levels. The system then generates a summary of scores and a graphical representation of each student's progress.
• AI and LLM-based systems – a new generation of tools based on artificial intelligence, including LLMs such as ChatGPT, enables personalised feedback, supports self-reflection, and fosters metacognitive development. Recent research shows that LLMs are effective in assessing student work (Floden, 2024), as well as in assisting with problem-solving and project-oriented tasks (Divjak et al., 2025).

4. Advantages of Automated Assessment

5. Limitations and Challenges

6. Case Studies

Below are two case studies involving automated assessment, prepared on the basis of scientific research.

7. Can ChatGPT Grade Student Exams?

Context

With the growing administrative and evaluation workload in higher education, interest in automated assessment is increasing. In this context, Swedish researcher Jonas Flodén (2025) poses a practical question: can artificial intelligence – specifically ChatGPT – reliably grade student exams in higher education?

In one university course in Sweden, Flodén analyzed 463 student answers to open exam questions, which were then assessed by human assessors and by ChatGPT (version 3.5). Each answer was graded multiple times, resulting in a total of 1,389 grades. The aim was to compare the accuracy, consistency, and assessing tendencies of humans and artificial intelligence.

Key Findings

• In 70 % of cases, ChatGPT gave a grade within ±10 % of the expected human grade, and in 31 % of cases, within ±5 %.
• The AI avoided extreme grades (very high or very low) and slightly favored scores that were above average.
• The AI performed better on general questions but struggled with questions requiring specific details from lectures.
• Teachers were surprised by the alignment of the grades but expressed concern about the lack of transparency in the criteria used by the AI.

Reflection Questions:

• In which situations would the use of AI for grading be acceptable in your course?
• Can you imagine using AI for the initial assessment of student essays, followed by human review?
• How would you explain the assessment criteria to students if the grade is partially generated by AI?

8. E-assessment in Mathematics – What Are Students’ Attitudes?

Context

Given the increasing use of digital tools in teaching, the question of transparent and reliable assessment is becoming crucial. In the paper E-assessment in mathematics in higher education: a student perspective, researchers Divjak, Žugec, and Pažur Aničić (2022) examine students’ attitudes toward an automated e-assessment system in university-level mathematics courses.

The study was conducted with 631 students during three different stages of the semester. Surveys and online assessments were used within higher education mathematics courses. The focus was placed on four key aspects:

• transparency and fairness in assessment
• transparency in two-phase assessment – formative + summative with feedback
• educational value of the technology, i.e., whether the tool has a meaningful pedagogical purpose
• alignment with the complexity of learning outcomes

Students generally supported e-assessment, provided that the criteria were clear, communication structured, and the technology meaningfully integrated into the learning process.

Key Findings

• Perception of fairness and transparency: Students perceived e-assessment as fair and reliable, especially when they understood the criteria and could see how the system worked.
• Formative support: The systems enabled continuous feedback, reducing exam-related stress.
• Pedagogical relevance: The tools improved understanding of course content, but only when the assessment results had educational value – not when they were perceived as “just another grade”.
• Handling complex outcomes: The systems handled routine tasks well, but students reported insufficient support when solving multi-step problems and more complex tasks.

Reflection Questions:

How would you ensure that digital tools are transparent and understandable for students?

How can formative feedback be incorporated into automated assessment systems?

How would you monitor and assess more complex tasks in mathematics or similar disciplines?

9. Activity for Participants

While reading this book and the accompanying materials:

• create short notes summarising the key advantages and challenges of automated assessment
• prepare one critical question or comment for discussion (e.g., “Can an automated system accurately assess the creative contribution in essay writing?”)

10. Conclusion and Recommendations for Teachers

Automated assessment can significantly simplify course organisation and provide valuable insights.

However, it is important that:

• the technology is pedagogically justified and purposeful
• assessment is aligned with the learning outcomes
• students understand the assessment criteria and procedures
• tools based on LLMs serve as a complement, not a replacement, for teacher judgement

By integrating automated methods into teaching, efficiency and student engagement can be increased, but it is at the same time essential to remain critical and ethically responsible in the use of automated assessment.

11. Self-Assessment and Reflection Questions

Self-assessment questions

1. Explain what is meant by the term automated assessment.

2. Provide an example of a technology or tool used for automated assessment and describe its basic function.

3. Describe the main advantages of automated assessment for teachers and students.

4. List the most common challenges and limitations of automated assessment in higher education.

5. Explain how automated assessment can be used for formative purposes, not only for summative assessment.

Reflection questions

1. Consider in which situations automated assessment would be useful in your course, and when human judgment should still play a key role.

2. Describe how you would explain the criteria and operation of an automated assessment system to students to ensure transparency and trust.

3. Suggest an example of how you could combine automated and teacher-based assessment in one activity (e.g., a quiz + an essay with a rubric).

4. Consider how you could use learning analytics and visualisations (e.g., Moodle progress reports) to monitor and support students.

5. Evaluate which ethical and pedagogical measures you would implement if you use an AI tool (e.g., ChatGPT) in the process of assessing student work.

12. Sources and Further Reading

 Clark, I. (2012). Formative assessment: Assessment is for self-regulated learning. Educational Psychology Review, 24(2), 205–249. 

Divjak, B., Žugec, B., Pažur Aničić, K. (2022). E-assessment in mathematics in higher education: A student perspective. International Journal of Mathematical Education in Science and Technology. 

Divjak, B., Svetec, B., Horvat, D. (2025). Generative AI in Mathematics Education: Analysing Student Performance and Perceptions over Three Academic Years. International Journal of Technology Enhanced Learning. In the process of being published.

Ifenthaler, D. (2023). Investigating self-assessment use in higher education with learning analytics. Journal of Computer Assisted Learning, 39(1), 222–233. 

Flodén, J. (2025). Grading exams using large language models: A comparison between human and AI grading of exams in higher education using ChatGPT. British Educational Research Journal. 

Khasawneh, M. A. S., Aladini, R., Assi, N., Rawashdeh, H. (2025). The effect of using artificial intelligence-based feedback in electronic portfolios on EFL students’ academic emotion regulation and mindfulness. Language Testing in Asia, 15(1), 10. 

Kumar, H., Xiao, R., Lawson, B., Musabirov, I., Shi, J., Wang, X., Luo, H., Williams, J., Rafferty, A., Stamper, J., Liut, M. (2024). Supporting self-reflection at scale with large language models: Insights from randomized field experiments in classrooms. 

Panadero, E., Jonsson, A. (2013). The use of scoring rubrics for formative assessment purposes revisited: A review. Educational Research Review, 9, 129–144. 

Ramesh, D., Sanampudi, S. K. (2021). An automated essay scoring systems: A systematic literature review. Artificial Intelligence Review, 55, 2495–2527.