Data literacy and artificial intelligence – biases and limitations

3. Consequences of bias in the educational use of artificial intelligence

The following outlines the most important consequences of bias in the educational application of artificial intelligence and their possible effects on the quality and fairness of the educational process.

Examples of Situations of Bias and Unfair Outcomes in Education

Example 1 – Analysis of Written Assignments

For example, students who write more creatively, use non-standard expressions, or deviate from the prescribed format may receive lower grades, even when the content of their work is high quality and relevant. In this way, the algorithm does not recognise the complexity and diversity of academic expression, which can result in unfair outcomes and demotivate students.

Example 2 – Monitoring Participation in Online Classes

For example, a student who carefully studies materials outside the platform, takes their own notes, or collaborates with colleagues through other digital channels may be incorrectly labeled as “inactive.” Such bias in measuring engagement leads to inaccurate conclusions about students’ actual activity and can affect decisions about planning and adjusting classes or assessing student performance.

Example 3

Situations in which the system unknowingly favours students with better technical resources, stable internet access, or higher levels of digital skills, while students who face technical limitations or have different learning styles may be incorrectly assessed as less engaged.

Example 4 

If the algorithm makes recommendations based on previous results, students with lower initial success may be given easier tasks and thus have fewer opportunities to progress. This does not reduce existing differences, but further deepens them.

Example 5

From a legal perspective, the risks relate to the protection of personal data and student privacy. Irresponsible collection, storage, or processing of data may conflict with the General Data Protection Regulation (GDPR). Particular attention should be paid to informed consent, limitation of the purpose of processing, and minimisation of the amount of data collected. For example, if data on students’ behaviour in the digital environment is used for automated evaluation without their consent or the possibility of insight into the decision-making process, this violates the principle of transparency and threatens students’ right to fair treatment. For this reason, it is important for higher education institutions to develop clear data management policies, provide regular education on the ethical use of AI, and introduce ongoing human oversight of algorithmic systems. It is the only way to maintain fairness, transparency, and accountability in the educational process.

These examples demonstrate how biases in AI systems can have real and measurable consequences for the educational process. It is therefore crucial that teachers and educational professionals develop the ability to critically analyse the results generated by such systems, understand their limitations, and ensure that technology is used in a way that promotes fairness, transparency, and inclusiveness in education.