Assessment of the Implementation (Orchestration) of Learning Design for a Targeted Group of Students

1. Overview of Key Concepts and Evaluation Activities

Successful implementation of learning design involves more than mere technical execution – it includes the thoughtful orchestration of all teaching activities, resources, and interactions, tailored to a specific group of students. Evaluation of implementation allows us to examine the extent to which the learning design has been realized in a real teaching context and what impact it has had on student learning.

Learning design may have been developed using different models and supported by suitable learning design tools. Particularly prominent models include: ABC, BDP, and OULDI.

Subsequently, this design was (automatically) transferred into the learning management system (LMS) used by your higher education institution, most commonly a version of Moodle. The initial structure was then enriched with learning and assessment materials, thereby preparing the course for students.

During the implementation phase of the learning design, the suitability of the design for the targeted group of students enrolled in the course is monitored. Student progress is tracked using learning analytics, which is based on the collection of data during the learning and teaching process, its analysis, and the presentation of results to teachers and students. Whenever possible, the presentation of results should be accompanied by an appropriate interpretation of learning analytics, and adequate training for teachers and students should be ensured.

It should be noted that not all data about student learning activities are available as digital traces within digital systems. However, systems used by higher education institutions provide highly valuable data, the analysis of which can yield important insights into how students learn and how teachers teach. In some cases, certain analyses are automated within the systems, but more sophisticated analyses and insights require knowledge of learning analytics (such training is also available within CARNET’s professional development courses under the title: Learning Analytics). This latter aspect is part of the process through which evaluations are made accessible to those who did not directly produce them but who nevertheless need to understand, at least in principle, all stages of the evaluation process.

Such analyses and insights form the basis for decision-making at the level of individual courses, activities, and more broadly at the level of higher education institutions or even the education system as a whole. Decisions are made by students, teachers, and responsible stakeholders at different levels of governance.

In this material, we will focus on learning analytics that support students and teachers, and for which data are automatically collected in learning design tools and learning management systems.

In the following section, three case studies on the evaluation of learning design implementation will be presented, which may influence the redesign of assessment, learning, and teaching activities, or the provision of feedback to students.

Learning Design Cycle