Learning Analytics for Improving Assessment

Introduction to Approaches and Application

In a digitalized educational environment, an increasing amount of data is available on student learning, their interaction with learning materials, activities, and learning management systems (LMS). Such data becomes the basis for learning analytics, and when the focus shifts to data related to assessment, we refer to assessment analytics.

Assessment analytics involves the collection, analysis, and interpretation of data to improve assessment practices, feedback, and ultimately, the achievement of learning outcomes.

Data used in assessment analytics may include results of formative and summative assessments, patterns of attempts and errors when solving tasks, the time spent completing tasks, as well as feedback provided by teachers to students and vice versa—feedback students provide regarding all aspects of assessment.

Key Elements of Learning Analytics for Assessment

1. Data Sources

Data for assessment analytics is collected from various sources:

• • LMS systems that provide an overview of assessments and question types (e.g., Moodle, Canvas)
  • digital tests and quizzes (e.g., in Moodle, Kahoot)
  • activities such as essays, discussions, and portfolios submitted in physical form
  • tracking of time and number of attempts (e.g., ISVU, LMS)
  • feedback from teachers and students (e.g., institutional surveys or course-level questionnaires)

2. Ethical Challenges and Data Quality

The collection and use of data face numerous ethical challenges, including locating and interpreting data, informed consent, privacy and de-identification of data, and data classification and management (Slade & Prinsloo, 2013). Data quality is crucial for the validity of further analysis and requires systematic attention to:

• relevance – whether the data is related to assessment goals

• accuracy and completeness – whether the data is precise and representative

• timeliness – whether the data is available when needed

• ethics – whether the data is collected and used responsibly

3. Tools and Methods of Analysis

Assessment analytics include descriptive (what happened), diagnostic (why it happened), predictive (what is likely to happen), and prescriptive (what should be done) approaches (Siemens & Long, 2011).
Common tools for assessment analytics include:

• visualizations through dashboards
• statistical analyses (e.g., correlations, regressions)
• machine learning for predicting success

Figure 1: Example of a student dashboard

For example,

simple visualizations can reveal a mismatch between the time spent on a task and the final grade, indicating a potential need to adjust assessment methods.

4. Examples of Applying Assessment Analytics

1. Monitoring student progress in quizzes (formative analytics)
   The teacher uses the Quiz statistics report in Moodle to analyse success by question.
   For example, if analytics show that more than 70% of students are making the same error, the teacher may identify unclear wording or concepts that need reinforcement. Students simultaneously receive feedback on topics they need to review, enabling targeted learning.

2. Identifying students at risk of failure
   In a course with multiple assessment stages (quiz, essay, final exam), the teacher analyses patterns of participation and performance.
   For example, if a student submits late and has a low number of quiz attempts, the system may generate an alert. The teacher can then intervene and offer support (consultations, additional materials). This approach is widely known as early warning analytics.

3. Comparing assessment consistency (reliability analytics)
   In a course where multiple teachers assess essays using a rubric, analytics can calculate discrepancies in scoring.
   If differences exceed 20%, recalibration of the criteria may be needed. This type of analytics ensures fairness and objectivity (see Divjak et al., 2023).

4. Feedback analytics
   Text analysis of teacher and student comments (e.g., in Workshop or Assignment activities) can show which types of feedback dominate—descriptive, motivational, or critical.
   For example, if analytics show most feedback contains only “good” or “OK”, the teacher may organize a session on giving constructive feedback. Such analyses can be performed using AI language models combined with tools like NVivo.

5. Implications for practice and decision-making

Learning analytics can support teachers and decision-makers in:

• • identifying ineffective or non-objective assessment forms
  • monitoring the achievement of learning outcomes
  • individualizing support for students
  • informing curriculum design and student support measures
  • detecting students at risk of dropout
  • checking the reliability and validity of the assessment (Divjak et al., 2023)

Figure 2: Implications of learning analytics

5. Interpreting learning analytics results

Proper data interpretation enables evidence-based decision-making, which is particularly crucial in high-stakes assessment contexts. Some interpretations require expertise, especially when predictive algorithms are involved, so institutional support for teachers is necessary. If dashboards are provided to students, explanations of visual representations must be clear.

Although analytics offer powerful tools, they must be used responsibly. Risks include misinterpretation, student labelling, and excessive automation. It is recommended to involve students in discussions about data use and ensure transparency and purposefulness (Ifenthaler & Yau, 2020).

External Resource

For additional understanding of learning analytics in the age of artificial intelligence, watch the following video lecture:

Dragan Gašević: Towards Novel Assessment with Learning Analytics for the Age of AI, Tecnológico de Monterrey

References:

• Divjak, B., Svetec, B., Horvat, D., Kadoić, N. (2023). Assessment validity and learning analytics as prerequisites for ensuring student-centred learning design. British Journal of Educational Technology, 54, 313–334.
• Ifenthaler, D., Yau, J. Y.-K. (2020). Utilising learning analytics to support study success in higher education: A systematic review. Educational Technology Research and Development, 68(4), 1961–1990.
• Siemens, G., & Long, P. (2011). Penetrating the fog: Analytics in learning and education. EDUCAUSE Review, 46(5), 30–40.
• Slade, S., & Prinsloo, P. (2013). Learning analytics: Ethical issues and dilemmas. American Behavioral Scientist, 57(10), 1510–1529.

Task for Participants

Aim: Identify opportunities for using learning analytics to enhance your own assessment practice.

Instructions:

1. Analyse an example (or use your own course):
   Select one Moodle activity (e.g., quiz, assignment, workshop) and use available reports (Activity report, Quiz statistics, Grades overview) to detect a pattern or issue (e.g., high error rate, late submissions, low activity).
   Describe what these data might indicate for improving assessment.
2. Propose an improvement:
   Based on the identified data, suggest a specific change in your course (e.g., modifying quiz questions, changing deadlines, adding content, or training students on criteria).
3. Reflect:
   In a few sentences, explain how you could incorporate learning analytics long-term to monitor students, plan assessments, and improve course quality.