Orientation: Case studies in learning analytics

Site:	Loomen za stručna usavršavanja
Course:	Learning Analytics
Book:	Orientation: Case studies in learning analytics

Printed by:	Gost (anonimni korisnik)
Date:	Wednesday, 29 July 2026, 3:15 AM

Description

Review the abstracts of three studies related to learning analytics, then select one article to read and study in full.

1. Predictors of summative assessment results
2. Providing feedback
3. Dashboard and prescriptive analytics

1. Predictors of summative assessment results

Context:
Faculty of Organization and Informatics, University of Zagreb; courses Mathematics 1 (undergraduate programme) and Discrete Structures with Graph Theory (graduate programme). For both courses, the learning design was developed using the BDP tool (constructive alignment), and innovative learning and teaching strategies are applied.
Objective:
The objective is to compare the predictive power of formative assessment results (quizzes, homework assignments) and other student activities (videos, e-books, class attendance) for summative assessment outcomes (midterm exams) and to develop a reliable predictive model.
Data:
Data on assessment results (formative — quizzes, homework assignments; summative — midterm exams); activity records (logs) in the LMS (videos, e-books); class attendance data; responses to a student survey; a total of 813 students across two consecutive academic years (2021/2022 and 2022/2023).
Analysis:
A Random Forest machine learning algorithm (implemented in R) was used for predictions, including testing the efficiency of different models; distribution of students into classes and transitions between classes; analysis of student logs by class; and descriptive statistics for the student survey.
Results:
Formative assessment, together with previous summative assessment, proved to be a stronger predictor of summative assessment outcomes than other student activity data. The study highlighted the importance of data completeness and quality, as well as a clear alignment between assessment and learning outcomes when predicting student performance. Predictions were found to be less reliable for students with the lowest and highest results. It was also observed that other factors may influence predictions, such as the level of learning outcomes or factors that are difficult to capture from digital data, including the learning environment and individual learning strategies.
Full study:
Divjak, B., Svetec, B., & Horvat, D. (2024). How can valid and reliable automatic formative assessment predict the acquisition of learning outcomes? Journal of Computer Assisted Learning, 40(6), 2616–2632.

2. Providing feedback

Context:
Computer Science School at an Australian university; the two largest programmes are the graduate Data Science programme and the undergraduate Computer Science programme.
Objective:
The objective is to examine the alignment of instructors’ feedback practices in real educational settings with well-established theories of learner-centered feedback, while analysing differences between two large datasets (undergraduate and graduate programmes).
Data:
Data on feedback provided through the LMS (2022): a total of 95 courses across all years of study, involving 4,959 students and approximately 200 instructors. The selected programmes represented the largest student populations at each level of study, with 2,187 graduate and 2,772 undergraduate students. Ten percent of the feedback from each course was analysed, amounting to 16,408 sentences.
Analysis:
Descriptive and inferential statistics; Ordered Network Analysis (implemented in R).
Results:
The study shows that feedback practices in higher education, as analysed within the selected study programmes, are partially aligned with the learner-centered feedback model, with a pronounced focus on the sensemaking dimension. However, significant differences were observed in the other two dimensions — future impact and agency — indicating different approaches depending on the level of education. At the graduate level, future impact is prioritised, with guidance for advanced tasks. At the undergraduate level, greater emphasis is placed on fostering agency through student engagement and participation. The study also shows that feedback practices differ according to levels of student achievement.
Full study:
Aldino, A. A.; Tsai, Y.-S.; Gupte, S.; Henderson, M.; Nath, D.; Gašević, D.; Chen, G. (2025).
Analytics of Learner-Centered Feedback: A Large-Scale Case Study in Higher Education . Computers & Education, 237, 105360.

3. Dashboard and prescriptive analytics

Context:
Faculty of Business and Law; a learning analytics dashboard developed with students using a participatory approach. The dashboard includes descriptive, predictive, and prescriptive elements.
Objective:
The objective is to collect and analyse undergraduate students’ perceptions of the impact of using a learning analytics dashboard, focusing on its usefulness and its influence on student motivation during online and distance learning.
Data:
A sample of 30 students who used the dashboard for 4–15 weeks; data collected through a survey and interviews in which students provided feedback on the dashboard and suggestions for improvement; system activity data (logs).
Analysis:
Descriptive and inferential statistics for the analysis of closed-ended survey questions and log data; thematic analysis of responses to open-ended survey questions and interviews.
Results:
The dashboard was shown to have a positive impact on student behavior and performance. Although all dashboard functionalities were perceived as useful, particular value was attributed to prescriptive elements, especially recommendations for learning materials and the ability to contact instructors and university support services. The value of providing clear suggestions for activities that can improve success was emphasised, as well as the impact on motivation and self-regulated learning. Based on the study, further improvements to the dashboard are planned.
Full study:
Herodotou, C.; Carr, J.; Shrestha, S.; Comfort, C.; Bayer, V.; Maguire, C.; Lee, J.; Mulholland, P.; Fernandez, M. (2025). Prescriptive analytics motivating distance learning students to take remedial action: A case study of a student-facing dashboard, 15th International Learning Analytics and Knowledge Conference (LAK 2025), March 03–07, 2025, Dublin, Ireland. ACM, New York, NY, USA.