Video lesson: Institutional Implementation and Evaluation of Learning Design

In this video lesson, we will learn how learning design is evaluated, how it is used at the institutional level, and what this looks like in practice at the Open University in the United Kingdom. Institutions that systematically apply learning design achieve greater alignment of study programmes with labour market needs, as well as with the personal needs of students. They provide better support to teachers and ensure a higher-quality student experience. The evaluation of learning design can be carried out in several ways. For example: through learning analytics based on institutional data, through focus groups with students, teachers, and external stakeholders such as employers and experts, who provide feedback and suggestions for improvement, through periodic surveys of students, teachers, and external stakeholders using online questionnaires, which help track trends over time and identify changes in perceptions of the programme quality, through internal and external quality assurance procedures of the study programme, where the achievement of learning outcomes at both the programme and course levels is specifically examined. By applying selected procedures, the quality of teaching at the institutional level is continuously improved. To discuss this topic, we spoke with Professor Bart Rienties from the Open University in the United Kingdom, an institution that is a global leader in the use of learning design and learning analytics. Interview BLAŽENKA DIVJAK: Hello, Bart, please tell us your name and affiliation. BART RIENTIES: Hi, I’m Bart Rienties. I am a professor of learning analytics at the Open University in the United Kingdom. BLAŽENKA DIVJAK: Thank you. So the first question is: What was the institutional policy implementation of learning design at your institution, as well as a systematic approach to the use of digital technologies and tools in teaching and learning? BART RIENTIES: That's a great question. At the Open University we teach at scale. So we teach around 200,000 students somewhere on this planet. So that means that we really have to think very meticulously and structurally in terms of how we design our learning experiences for our students. Since 2008, we've adopted the OU Learning Design Initiative, where we looked at: Okay, how can we start to map all the amazing learning activities that our teachers make? And that approach has been adapted and adjusted over the last 15 to 17 years into a very systematic way of working. That systematic way of working is captured in a very detailed way in how we produce our courses using digital technology. We are a digital-technology university, so in principle, all our learning activities are designed in an online manner. BLAŽENKA DIVJAK: Thank you. Have you found that teachers who plan activities in line with learning outcomes have more engaged students? Have you done any research on how learning design affects student results? BART RIENTIES: It is really important when we work with teachers on their learning design that they carefully plan what they expect their students to achieve in terms of learning outcomes — and whether those learning outcomes are realistic for students. As I mentioned before, we teach most of our students online; they come from all kinds of different backgrounds. So teachers need to really carefully plan for diversity. Over the last 15 years, we’ve done a lot of research in terms of: Which design decisions have a strong impact on student satisfaction? Which learning-design activities strongly support students in successfully completing tasks, modules, and ultimately qualifications? We know from our research that around two-thirds of all student engagement is determined by how we design our courses. It's really important to critically review, together with teachers and students, how to best design our courses for our learners. BLAŽENKA DIVJAK: Thank you. And finally, what recommendations would you give to higher education institutions, teachers, and students based on your research? Should they be continuously trained for teaching and learning? If yes, how? BART RIENTIES: Great question. So my biggest recommendation, based not only on the research we’ve done in the past with Aldi, but also on various projects with the University of Zagreb and Ireland — the most recent ones — is this: If we were using the same tools we developed in 2008 today, we would have missed so many opportunities to further strengthen learning design. The way we look at learning design has fundamentally changed in the last five or six years. There are so many amazing ways teachers can test and experiment — for example: Can I teach week-three mathematics in a slightly different way for this particular group of students? What we’ve found is that we need to give teachers space to experiment. And with the learning-design features that we’ve developed together with many universities across the globe — and of course with the University of Zagreb — this BDP tool allows for amazing opportunities for teachers to see what the implications are if they change something in week three. That is really important! It’s something unique that we didn’t have before. And regarding continuous training: if you are curious as a teacher, you naturally want to continuously improve what you are doing with your students. For some teachers, this comes naturally. For others, they may need a little dashboard to start with, something that says: Hey, maybe in week three of your course something doesn’t work entirely — here are two or three suggestions. So yes — let’s keep fine-tuning our teaching practices because they are so important for us to use. Conclusion The example of the Open University shows that a systematic approach to learning design and to the evaluation of learning and teaching processes brings concrete results, and greater satisfaction for everyone involved. Learning design should be an institutional practice. Assessment should be regular. Technology is a tool for support, not a goal in itself. Ultimately, nothing works without the continuous professional development of teachers and other relevant educators. Which elements could you apply immediately in your own institution? Thank you, and let’s continue learning.