Reading: Data-based decision-making

1. Introduction

The use of data as a basis for making educational decisions is not a new topic, with examples dating back to the last decade of the 20th century (Agasisti & Bowers, 2017). However, in the era of highly digitalised and technologically supported higher education, shaped by the challenges of the pandemic and the revolution of generative artificial intelligence (GenAI), higher education institutions have access to larger volumes of diverse student data than ever before (Prinsloo, 2020). In learning management systems (LMS) (e.g., Moodle), information systems with sociodemographic and enrolment data on students (e.g., ISVU), and other information systems in (higher) education, large amounts of data are generated on a daily basis (Gašević et al., 2019; Nguyen et al., 2020). There is also potential for using less typical types of data, such as geospatial data (Valčík, 2012).

Here we show how data and analytics can be used — and how they are used — in higher education to support decision-making at different levels, with the aim of improving the quality of learning and teaching and supporting the management of higher education institutions.

2. Support for teaching and learning decisions

Data analysis of information generated through students’ interaction with educational technologies has been recognised as a valuable approach that enables better insights and a deeper understanding of learning processes. This has encouraged the development of strategic, data-informed decision-making in education, which was crucial for the emergence of a new field of research and practice — learning analytics (Gašević et al., 2019).

Definition. For decades, teachers have monitored student progress, designed assessments, and used evidence to improve learning. However, learning analytics brings novelty and enhances these practices through the use of new digital data and analytical methods from data science and artificial intelligence.

Benefits and challenges. Learning analytics can be based on different types of data, which will be discussed in the following sections of the e-course. The emphasis is on data generated through interactions between students and information technology, primarily learning management systems (Divjak & Maretić, 2017). Data-driven learning analytics enables the creation of insights, decisions, and actions aimed at improving learning and teaching (Chatti et al., 2020). Insights into students’ learning processes and learning difficulties help teachers identify weaknesses in learning and teaching activities and provide constructive feedback that can guide and improve further learning (Gašević et al., 2015). Furthermore, learning analytics supports personalised learning (Chatti & Muslim, 2019) and the development of flexible learning pathways (Tsai et al., 2018).

In addition to improving the quality of learning experiences, learning analytics helps identify factors influencing student success and completion rates (Tsai et al., 2018), thus playing an important role in identifying students at risk of dropping out and in planning interventions aimed at maximising student retention (Susnajk et al., 2022). Learning analytics can also provide useful insights directly to students themselves by supporting self-regulated learning (Schumacher & Ifenthaler, 2018). Learning analytics, therefore, supports data-informed decision-making at multiple levels: from the institutional level to the level of teachers and, ultimately, students.

Despite the significant increase in interest among higher education institutions in using learning analytics to improve learning and teaching, as well as growing funding opportunities, research has shown that the implementation of learning analytics in higher education is still not systematic or widespread, and that challenges remain for effective data use (Tsai et al., 2018). There are various reasons for this, such as the lack of policies and strategic support, insufficient capacities for data-driven decision-making, and differing levels of stakeholder engagement (Tsai & Gašević, 2017), which affect the usefulness of learning analytics tools and the relevance of policies (Tsai et al., 2022), as well as institutions’ experience with learning analytics (Tsai et al., 2021). Furthermore, the challenge of trust in learning analytics is also present (Tsai et al., 2021).

An additional key challenge in creating the value of learning analytics is data quality, as the quality of outcomes largely depends on the quality of input data (Sheikh et al., 2022). Bias and unfairness that may arise from data affect the trustworthiness of learning analytics, especially when it comes to predictive models (Li et al., 2022).

Finally, even after adoption and significant investments in learning analytics, some universities have failed to achieve their strategic goals. It is therefore emphasised that using learning analytics to achieve strategic value requires investment not only in infrastructure, but also in staff professional development and strategic planning (Sheikh et al., 2022).

Relevant knowledge and skills related to strategic planning can be acquired in the first e-course of the programme “Management of the Digital Transformation of Educational Institutions”, entitled Strategic Planning for Digital Technology Implementation.

Connection with related fields. There are several “sister” research fields with which learning analytics shares similar goals. One of them is Educational Data Mining (EDM). Similar to learning analytics, EDM applies machine learning, data mining algorithms, and statistics to educational data (Dutt et al., 2017). However, EDM is more focused on technology and models, whereas learning analytics is more focused on learning itself (Sghir et al., 2023), although the two fields overlap more often than they differ (Rienties et al., 2020). Another field closely related to learning analytics is discussed in the following section.

3. Support for operational decision-making

The benefits of data analytics in education go beyond learning and teaching processes, and data on students and stakeholders at the administrative level can also provide important insights. A field with a different focus, but related to learning analytics, is academic analytics.

Definition.

The results of academic analytics are more often used by those in management positions (Agasisti & Bowers, 2017).

4. Conclusion

For this data to become meaningful, thoughtful analyses are essential as support for decision-making. Learning analytics plays a key role in this process by providing a foundation for decisions related to learning and teaching. It is also important to mention academic analytics, which is more oriented toward operational decision-making.

Figure 1: Key benefits of learning analytics

5. Questions for self-assessment and reflection

             How does learning analytics contribute to decision-making at different levels (institution, teachers, students)?

             Which barriers hinder the systematic implementation of learning analytics in higher education, and how can they be mitigated?

6. Literature

1. Agasisti, T., & Bowers, A. J. (2017). Data analytics and decision making in education: towards the educational data scientist as a key actor in schools and higher education institutions. In Handbook of Contemporary Education Economics. Edward Elgar Publishing. 
2. Chatti, M. A., & Muslim, A. (2019). The PERLA Framework: Blending Personalization and Learning Analytics. The International Review of Research in Open and Distributed Learning, 20(1). 
3. Chatti, M. A., Muslim, A., Guesmi, M., Richtscheid, F., Nasimi, D., Shahin, A., & Damera, R. (2020).How to Design Effective Learning Analytics Indicators? A Human-Centered Design Approach (pp. 303–317).
4. Divjak, B., & Maretić, M. (2017). Learning Analytics for Peer-assessment. Journal of Information and Organizational Sciences, 41(1), 21–34. 
5. Dutt, A., Ismail, M. A., & Herawan, T. (2017). A Systematic Review on Educational Data Mining. IEEE Access, 5, 15991–16005. 
6. Ferreira, S. A., & Andrade, A. (2016). Academic analytics: Anatomy of an exploratory essay. Education and Information Technologies, 21, 229–243. 
7. Gasevic, D., Tsai, Y.-S., Dawson, S., & Pardo, A. (2019). How do we start? An approach to learning analytics adoption in higher education. The International Journal of Information and Learning Technology, 36(4), 342–353. 
8. Goldstein, P. J. (2005). Academic Analytics: The Uses of Management Information and Technology in Higher Education.
9. Li, W., Sun, K., Schaub, F., & Brooks, C. (2022). Disparities in Students’ Propensity to Consent to Learning Analytics. International Journal of Artificial Intelligence in Education, 32(3), 564–608. 
10. Nguyen, A., Gardner, L., & Sheridan, D. (2020). Data Analytics in Higher Education: An Integrated View. Journal of Information Systems Education, 31(1), 61–71. 
11. Prinsloo, P. (2020). Of ‘black boxes’ and algorithmic decision-making in (higher) education – A commentary. Big Data & Society, 7(1), 205395172093399. 
12. Rienties, B., Køhler Simonsen, H., & Herodotou, C. (2020). Defining the Boundaries Between Artificial Intelligence in Education, Computer-Supported Collaborative Learning, Educational Data Mining, and Learning Analytics: A Need for Coherence. Frontiers in Education, 5. 
13. Schumacher, C., & Ifenthaler, D. (2018). Features students really expect from learning analytics. Computers in Human Behavior, 78, 397–407. 
14. Sghir, N., Adadi, A., & Lahmer, M. (2023). Recent advances in Predictive Learning Analytics: A decade systematic review (2012–2022). Education and Information Technologies, 28(7), 8299–8333. 
15. Sheikh, R. A., Bhatia, S., Metre, S. G., & Faqihi, A. Y. A. (2022). Strategic value realization framework from learning analytics: a practical approach. Journal of Applied Research in Higher Education, 14(2), 693–713. 
16. Society for Learning Analytics Research. (n.d.). 
17. Susnjak, T., Ramaswami, G. S., & Mathrani, A. (2022). Learning analytics dashboard: a tool for providing actionable insights to learners. International Journal of Educational Technology in Higher Education, 19(1), 12. 
18. Tsai, Y.-S., & Gasevic, D. (2017). Learning analytics in higher education --- challenges and policies. Proceedings of the Seventh International Learning Analytics & Knowledge Conference, 233–242. 
19. Tsai, Y.-S., Kovanović, V., & Gašević, D. (2021). Connecting the dots: An exploratory study on learning analytics adoption factors, experience, and priorities. The Internet and Higher Education, 50, 100794. 
20. Tsai, Y.-S., Moreno-Marcos, P. M., Jivet, I., Scheffel, M., Tammets, K., Kollom, K., Gašević, D., Hashim, F., Alam, G. M., Siraj, S., Macfadyen, L. P., Dawson, S., Mahroeian, H., Daniel, B., Okoye, K., Arrona-Palacios, A., Camacho-Zuñiga, C., Hammout, N., Nakamura, E. L., … Pardo, A. (2018). The SHEILA Framework: Informing Institutional Strategies and Policy Processes of Learning Analytics. Journal of Learning Analytics, 5(3), 31–47. 
21. Tsai, Y.-S., Singh, S., Rakovic, M., Lim, L.-A., Roychoudhury, A., & Gasevic, D. (2022). Charting Design Needs and Strategic Approaches for Academic Analytics Systems through Co-Design. LAK22: 12th International Learning Analytics and Knowledge Conference, 381–391. 
22. Valcik, N. A. (2012). Using Geospatial Information Systems for Strategic Planning and Institutional Research for Higher Education Institutions. International Journal of Strategic Information Technology and Applications, 3(4), 31–47.
23. van Barneveld, A., Arnold, K., & Campbell, J. (2012). Analytics in Higher Education: Establishing a Common Language.